Skip to content

WeeklyTelcon_20190910

Jump to bottom
Geoffrey Paulsen edited this page Sep 10, 2019 · 1 revision

Open MPI Weekly Telecon

  • Dialup Info: (Do not post to public mailing list or public wiki)

Attendees (on Web-ex)

  • Geoffrey Paulsen (IBM)
  • Brian Barrett (AWS)
  • Harumi Kuno (HPE)
  • Ralph Castain (Intel)
  • Todd Kordenbrock (Sandia)
  • Josh Hursey (IBM)
  • Erik Zeiske
  • Brendan Cunningham (Intel)
  • Edgar Gabriel (UH)
  • David Bernhold (ORNL)
  • Howard Pritchard (LANL)
  • Michael Heinz (Intel)

not there today (I keep this for easy cut-n-paste for future notes)

  • Akshay Venkatesh (NVIDIA)
  • Artem Polyakov (Mellanox)
  • Brandon Yates (Intel)
  • George Bosilca (UTK)
  • Jeff Squyres (Cisco)
  • Joshua Ladd (Mellanox)
  • Mark Allen (IBM)
  • Matias Cabral (Intel)
  • Matthew Dosanjh (Sandia)
  • Nathan Hjelm (Google)
  • Noah Evans (Sandia)
  • Thomas Naughton (ORNL)
  • Tom Naughton
  • Xin Zhao (Mellanox)
  • mohan (AWS)

Agenda/New Business

Erik opened Issue 6950

  • Erik opened this issue, but no one has replied yet.
  • Everyone please read this issue, and comment if you have thoughts.

Discussed xpmem / CMA

  • PR6844 - Want to test if this affects containers.
    • Worth the question, don't see any reason not to take this.
    • Jeff will review and add comments.
    • Howard will do some testing and talk to Charlie-cloud.

Infrastrastructure

Process enforcement bots

  • No update (Brian on vacation)

Submodule prototype

  • Merged --recurse-submodules update into ompi-scripts Jenkins script as first step. Let's see if that works.

Release Branches

Blockers All Open Blockers

Review v3.0.x Milestones v3.0.4

  • No new news

Review v3.1.x Milestones v3.1.4

  • PR6556 and PR 6621 should go to the v3.x release branches.
  • No new news

Review v4.0.x Milestones v4.0.2

  • ABI changes: https://github.com/open-mpi/ompi/issues/6949

    • Linkers are a bit smarter now and we should define our ABI better.
    • Help it work with the tool.
    • Looks like in this
    • We have Open MPI the package, then we have Open MPI and Open SHMEM libraries.
      • Our versioning is on the larger package, not really on library level.
      • Compatibility guarantees are confusing
      • We're letting OpenSHMEM add new functions, though not Open MPI.
      • this is confusing for folks.
      • Tearing this apart will be challenging.
    • Lets take this particular issue seriously.
    • It would be cool to have CI - Geoff signs up to find out more information about tools.
    • This is probably okay for v4.0.2.
    • We should

  • Geoffroy Vallee has a system setup to run cross-compatibility, and can report out which versions are failing. Ralph will forward info to devel-core.

  • Still have some issues; we expect to still have to do an rc2, e.g., https://github.com/open-mpi/ompi/issues/6932.

  • Discuss Issue 6568 - large messages overwhelm put

    • PR 6961 went into master - Nathan said it might help.
      • George commented it's a partial solution.
      • See if this fixes 6568, and if it does consider for v4.0.2
      • Hold off on pulling into v4.0.x until after rc2, for easier regression testing.
      • The other interfaces don't have as tight of constraints, and might not hit this.
    • This SHOULD stay as a blocker, since it ends in hang.
    • We need to look for a workaround.
      • Could disable put completely.
      • Could use an opal_unlikely check of message-size, and only then kick it back if the message size is too large.
    • OB1 tries put / get, and if these don't work, it falls back to send/recv.?
    • possibly a flaw in put itself.
    • Jeff will ask george what would be viable workaround, and identify.
      • Not signing up to implement.

  • PR6942 - ready to merge.

  • https://github.com/open-mpi/ompi/issues/6949 - Geoff (and others please review)

  • MTT failures in Generic Simple unpack on v4.0.x - segfaults, assertions.

    • DDT-unpack assertion on v4.0.x

  • See older weekday notes for prior items.

Review Master Master Pull Requests

  • Howard will test master to see if PR 6961 fixes Issue 6568 (large messages overwhelm put)
    • If it goes well, we can
  • PR 6844 - If Jeff gives the okay, Howard says we should merge this.
    • This does fix what container folks were seeing (having to disable CMA)
    • Trying to talk to each other through vader, will talk to each other (bypassing CMA)
    • XPmem doesn't care about memspaces, just the key to access virtual address space.
    • This is a good PR.
    • Is this for v4.0.x or just master?
      • Need to investigate if it changes datastructures that are exchanged.
    • PMIx did a think in v3.1.4 to extend the modex at some point, since just added it to existing one.
      • So this does it similarly, so shouldn't be an issue.
  • IBM's PGI test has NEVER worked. Is it a real issue or local to IBM.
  • nVidia bought PGI, perhaps someone there could take a look?
    • Akshay said he'd talk to a PGI person at nVidia to see.
  • Edgar mentioned that Mark Allen should rebase PR6756 and get that in to resolve an issue another customer is seeing.

CI status

  • Cray running into problems again. :frown:
    • Back on track.

v5.0.0

Depdendancies

PMIx Update

ORTE/PRRTE

Next face to face

MTT

  • IBM has to triage some failures on master and v4.0.x and some test build issues. Josh Hursey thought they might be accidentally mixing XLC and PGI compilers. Will investigate.
  • Cisco has a build failure to investigate.

Back to 2019 WeeklyTelcon-2019

Clone this wiki locally