2023-09-11 - KDB Project Call Meeting Minutes

[alvin-c-shih]

Date

Monday, September 11, 2023 - 9:30am ET / 2:30pm UK

// Second Monday of every month

Attendees

Fullname	Affiliation	GitHub Username
Peter Gyorok	KX Systems	@gyorokpeter
Alvin Shih	Morgan Stanley	@alvin-c-shih
Aaron Davies	Morgan Stanley	@adavies42
Jo Shinonome	ExeQution Analytics	@jshinonome
Ross Duffy	Morgan Stanley	@rdffy
James McLeod	FINOS	@mcleo-d

Meeting notices

• FINOS Project leads are responsible for observing the FINOS guidelines for running project meetings. Project maintainers can find additional resources in the FINOS Maintainers Cheatsheet.

• All participants in FINOS project meetings are subject to the LF Antitrust Policy, the FINOS Community Code of Conduct and all other FINOS policies.

• FINOS meetings involve participation by industry competitors, and it is the intention of FINOS and the Linux Foundation to conduct all of its activities in accordance with applicable antitrust and competition laws. It is therefore extremely important that attendees adhere to meeting agendas, and be aware of, and not participate in, any activities that are prohibited under applicable US state, federal or foreign antitrust and competition laws. Please contact [email protected] with any questions.

• FINOS project meetings may be recorded for use solely by the FINOS team for administration purposes. In very limited instances, and with explicit approval, recordings may be made more widely available.

Agenda

• (5 min) Convene, roll call, welcome new people.

• Display FINOS Antitrust Policy summary slide

• Review Meeting Notices (see above)

• (5 min) Approve previous meeting minutes.

  • Approved by acclamation.

• (30 min) kuki Deep Dive

  • Further discussion on how this would work in an enterprise.
  • Looking concretely at re package - q plugin for Google's re2 library.
    • https://kuki.ninja/package/re
    • https://github.com/jshinonome/re
  • dep.q allows each package to register unload handler.
    • .z.exit would call them to shut down helper threads / clean up resources.
    • Otherwise can trigger SIGSEGV, resulting in production alerts.
    • Kafka plugin cleans up using atexit(3).
      • https://github.com/KxSystems/kafka/blob/master/kfk.c
    • But there are times when you want to shut down helper threads and then start them up again, such as qclone.
    • So maybe worth considering this for kuki.
  • re package includes a *.so.
    • Enterprises often have laggards on old versions of q / Linux.
      • KX_VER=2 vs KX_VER=3
      • RHEL7 vs RHEL8 / Windows / MacOS / etc.
        • Most use Linux at work but may want Windows / MacOS for the laptop.
      • Would be good to allow different platforms to be recognized.
    • pip can build on download
      • But still have to ensure correct C++ compiler is picked up as well as sensitivity to versions of Boost C++ templates.
  • kuki registry compatible with NPM (which is a JavaScript ecosystem!)
    • KX is leaning hard into Python
    • NPM ecosystem chosen for kuki:
      • Simpler than PyPi since developer just needs to supply a single JSON file
      • Better UI.
      • kuki registry written from scratch.

• (5 min) kdb.ai

  • Would be interesting to see if there could be a tie-in with Regulatory Horizon Scanning.
    • https://github.com/synechron-finlabs/horizon-scanning-core

• (5 min) Ideas for technical-documentation/ ?

  • HDB querying gotchas (i.e. limitations of .Q.ps)
  • Perhaps highlight existing resources:
    • For example, "Dark Corners of Q":
      • https://github.com/quintanar401/DCoQ

• (5 min) OSFF coming November 1st

  • https://events.linuxfoundation.org/open-source-finance-forum-new-york/

• (5 min) AOB, Q&A, Adjourn.

WebEx info

Webex

• https://finos.webex.com/finos/j.php?MTID=me61719ca3c3295a7cc76762d78392bc3

Dial-in and Project Materials

• US: +1-415-655-0003 US Toll
• UK: +44-20319-88141 UK Toll
• Access code: 2704 417 0390
• GitHub Repos:
  • https://github.com/finos/kdb
  • https://github.com/finos/kdb-studio
• Wiki Page:
  • https://finosfoundation.atlassian.net/wiki/spaces/DT/pages/329383945/kdb+Project
• KDB Metrics on FINOS
  • https://metrics.finos.org/goto/033b15c80f0d08cab70f40f4358932f6

[adavies42]

Aaron Davies/Morgan Stanley

[rdffy]

Ross Duffy/Morgan Stanley

[jshinonome]

Jo/ExeQution Analytics 🦊

[gyorokpeter]

Peter Gyorok/KX

[mcleo-d]

James McLeod / FINOS

[gyorokpeter]

Unload handler in dep.q:

kdb/q/dep/dep.q

Line 165 in a4317d1

.finos.dep.registerUnload:{[name;unload]

[gyorokpeter]

https://github.com/quintanar401/DCoQ

[alvin-c-shih]

Alvin Shih / Morgan Stanley