Goal:

• Instantiate big data analytics framework using OOM with these helm charts.
• Instantiate framework in various cloud-regions (that support K8S) via ONAP "Support for K8S based Cloud regions".

Components that make Big data analytics framework (Initial list, more may be required as we start digging in)

• HDFS
• Apache Spark
• Kafka 
• OpenTSDB
• M3DB
• PNDA Deployment manager (if applicable)
• PNDA DB Cleanup 
• Various web services for GUI (e.g Grafana)
• Prometheus collection service

Try to leverage standard docker images as much as possible.

Leverage and adapt helm charts that are available in the wild 

• HDFS:  https://github.com/apache-spark-on-k8s/kubernetes-HDFS
• Spark: https://github.com/helm/charts/tree/master/stable/spark

• Kafka: https://github.com/helm/charts/tree/master/incubator/kafka
• Prometheus: https://github.com/helm/charts/tree/master/stable/prometheus / https://github.com/coreos/prometheus-operator

For others we may need to create helm charts ourselves from scratch. 

For some of PNDA components (GUI, Deployment manager, DB cleanup manager), we need to create both Docker images (Docker files) and helm charts (if they are applicable)

Adapting them to ONAP may require changes for following reasons:

For OOM reasons:  OOM provides guidance on modularity.  It is good and shall be adapted.

For multiple deployments in cloud-regions: It should be possible to deploy framework multiple times in the same cloud region. Environment (values) shall have enough flexibility to deploy multiple times -  a way to use namespace for various framework deployments in the same cloud region.  Need to ensure that all services that belong to the framework talk with service+namespace names. It shall be possible to select new docker images without modifying the helm charts. It shall be possible to have different requirements on ports they expose, cpu, memory and disk requirements across various deployments.

Validation:

• Use OOM to bring up big data framework.
• Use PNDA example applications (ZTT and other applications) to ensure that they work
• Use K8S Plugin to bring up two instances of big data framework in a cloud region.
• Use PNDA examples in both instances and ensure that they all work fine.