From Kenny Paul (LFN) to Everyone: (09:56)

#topic datalake
Guobiao Mo reviedes his slides

From Me to Everyone: (10:03)

Q: we could provide hooks into the 3 existing ELK stacks (LOG, POMBA, CLAMP) - most already have external facing nodeports for their elasticsearch containers - btw there are 5 elasticsearch, 5 cassandara, 14+ mariadb/myssql 1 mongodb, 3 postgres, 2 redis and one graphdb in onap
there is a mongodb container in nbi/extapi

From Alexander Vul to Everyone: (10:07)

Is this infrastructure meant to be deployed as part of the ONAP footprint?

From Keong Lim k00759777 to Everyone: (10:08)

since DMaaP is based on Kafka, couldn't this also be done by simply retaining all the messages in Kafka?

From Alexander Vul to Everyone: (10:08)

Kafka is not meant to handle the volume of data we are talking about here...

From Keong Lim k00759777 to Everyone: (10:09)

there are orgs that do exactly that already with kafka

From Alexander Vul to Everyone: (10:09)

I am sure there are...

From Catherine Lefevre to Everyone: (10:10)

@Denghui, Andy - agenda updated but have you planned a TSC Readout about NS domain / Internal NS Descriptor today?

From Michael O'Brien to Everyone: (10:11)

So datalake will be Elasticsearch as a service - we should discuss this

From Catherine Lefevre to Everyone: (10:11)

if not then I can remove this readout for this week

From Michael O'Brien to Everyone: (10:12)

datalake pods ideally will be native kubernetes charts not cloudify blueprints for el-alto

From Michael O'Brien to Everyone: (10:12)

datalake pods ideally will be native kubernetes charts not cloudify blueprints for el-alto

From Ramki Krishnan to Everyone: (10:12)

Fair point raised by Keong Lim on Kafka - Kafka has programmable buffering.

From Keong Lim k00759777 to Everyone: (10:12)

https://www.confluent.io/blog/publishing-apache-kafka-new-york-times/

From Michael O'Brien to Everyone: (10:13)

woudn't we need DCAEGEN2 deployed plus secondary dep-* pods to use datalake

From Srini Addepalli (Intel) to Everyone: (10:13)

Is this activity intend to be done as part of POMBA or DCAE?


s this activity intend to be done as part of POMBA or DCAE?

From Jason Hunt to Everyone: (10:14)

Wouldn’t we expect that operators already have data lakes?  Should we focus on how to enable them to link ONAP to their data lakes?

From Brian to Everyone: (10:14)

+1 Jason's comment

From Michael O'Brien to Everyone: (10:14)

+1 Jason - pluggable data lake - so this datalake is just an RI

From Eric Debeau to Everyone: (10:14)

Most of operators have deployed their own data lakes. I agree that we need to explain how to link ONAP with existing data lakes

From Fernando (Fred) Oliveira to Everyone: (10:15)

I agree, ONAP should enable the feeds to an external data lake

From Andreas Geissler (Deutsche Telekom) to Everyone: (10:16)

agree with Eric and Jason, but as ONAP Addon the component would be useful.

From Lingli to Everyone: (10:16)

+1

From Michael O'Brien to Everyone: (10:18)

More very interested on the Datalake as a Platform (pluggable) aspect of this project - the actual implementation is more of an example

From Brian to Everyone: (10:18)

can we do it like netbox for IPAM as a contrib ?

From Michael O'Brien to Everyone: (10:18)

yes, contrib is a good idea

From Mike Elliott to Everyone: (10:19)

+1 - contrib (optional)

+1 - contrib (optional)

From Me to Everyone: (10:23)

onap          onap-contrib-netbox-app-84f877fb76-qhc9t                      1/1       Running            0          9h
onap          onap-contrib-netbox-app-provisioning-8f2vk                    0/1       Error              0          9h
onap          onap-contrib-netbox-app-provisioning-gwtqt                    0/1       Completed          0          9h
onap          onap-contrib-netbox-app-provisioning-j6hn9                    0/1       Error              0          9h
onap          onap-contrib-netbox-nginx-5c85578697-lp2bd                    1/1       Running            0          9h
onap          onap-contrib-netbox-postgres-65f4bb6459-rc5k8                 1/1       Running            0          9h
the error is normal - the completed is the 2nd run
above https://git.onap.org/oom/tree/kubernetes/contrib

From Eric Debeau to Everyone: (10:28)

+1 for contrilb

From Ramki Krishnan to Everyone: (10:28)

+1 on contrib

From Eric Debeau to Everyone: (10:28)

I think we should share about contrib at TSC level

From Srini Addepalli (Intel) to Everyone: (10:29)

+1 on contrib. ONAP to external datalake bus as Kafka is good enough in my view.

From Pamela Dragosh to Everyone: (10:30)

At the very least, trial it with DCAE and see if it fits. If not, go from there.

From Srini Addepalli (Intel) to Everyone: (10:30)

It is very common practice in Industry to use Kafka as a bus.

From Arash Hekmat (Amdocs) to Everyone: (10:30)

A contrib is a module that is not part of OOM master Helm chart for ONAP install. A contrib module has a separate chart in OOM and has to be installed separately.

From Catherine Lefevre to Everyone: (10:31)

TSC - are you ok to vote +1 POC only for Dublin, part of DCAE; 0 no opinion, -1 need to more time to reflect and descope

From Me to Everyone: (10:32)

arash is right - DCAEGEN2 not compatable yet with native kubernetes charts via contrib

From Andreas Geissler (Deutsche Telekom) to Everyone: (10:32)

+1

From Lingli to Everyone: (10:32)

+1

From Chaker Al-Hakim to Everyone: (10:32)

#vote +1

From Stephen Terrill to Everyone: (10:32)

#vote: +1