DateComponents memory usage considerations

[hassila]

So we are looking at using much more of Foundation now thanks to the Swift rewrite - looking at some of the implementation, one thing that is concerned us is some of the choices of internal representation of data that is unnecessarily large.

To take just one obvious example (exluding the layout overhead for the optional enum, but just looking at data storage):

https://github.com/apple/swift-foundation/blob/main/Sources/FoundationEssentials/Calendar/DateComponents.swift

public struct DateComponents : Hashable, Equatable, Sendable {
...
    internal var _era: Int?
    internal var _year: Int?
    internal var _month: Int?
    internal var _day: Int?
    internal var _hour: Int?
    internal var _minute: Int?
    internal var _second: Int?
    internal var _nanosecond: Int?
    internal var _weekday: Int?
    internal var _weekdayOrdinal: Int?
    internal var _quarter: Int?
    internal var _week: Int?
    internal var _weekOfMonth: Int?
    internal var _weekOfYear: Int?
    internal var _yearForWeekOfYear: Int?
...

~960 bits

Using 64-bits (for most platforms) for the majority of these is just massive overkill in terms of storage space as they are well-defined entities and 8/16/32 bits are sufficient (without resorting to any bit-stuffing) by just appropriate declaration;

public struct DateComponents : Hashable, Equatable, Sendable {
...
    internal var _era: Int32?  // likely overkill
    internal var _year: Int32?
    internal var _month: Int8?
    internal var _day: Int8?
    internal var _hour: Int8?
    internal var _minute: Int8?
    internal var _second: Int8?
    internal var _nanosecond: Int32?
    internal var _weekday: Int8?
    internal var _weekdayOrdinal: Int8?
    internal var _quarter: Int8?
    internal var _week: Int8?
    internal var _weekOfMonth: Int8?
    internal var _weekOfYear: Int8?
    internal var _yearForWeekOfYear: Int8?
...

~192 bits

That's a 5x difference.

One can argue for how many instances one might use for this particular use case etc, but for a foundational library it'd be nice to have an efficient implementation (both in time&space) by default, especially when it can just be declared.

Perhaps this is just a singular instance, haven't gone through and reviewed everything, but it was a bit concerning so wanted to raise the discussion.

Without having all the details, if this is due to bridging to legacy (ahem) languages, maybe there are some other way?

[parkera]

This is primarily due to complexity of implementation right now. This one has an “obvious” solution of using some kind of enum backing to save space when only 1 or a small set of fields is used. However, when trying to implement this it quickly becomes a very complex state machine because every property is mutable. I think we can solve it, but there is a high likelihood the result will be a lot of duplicated code.

[hassila]

But even with existing layout just changing the declarations naively would give the 5x? Or I'm missing something?

[parkera]

No, I think you are right that we can do something simple here in the short term and leave the heroics for later. I've added the "good first issue" keyword.

[parkera]

Keep in mind that date components fields are used not just to store values but also add values. So, for example the day field is not just the day of month and obviously fits in 1 byte - it could be a very large value and valid to pass to the API. e.g., add 10,000 days to this date.

[parkera]

Here's a couple of places where we create date components instances with just a few fields, which would be good patterns to consider if we wanted to create some internal representation that optimized for common cases while still allowing outliers:

https://github.com/apple/swift-foundation/blob/main/Sources/FoundationEssentials/Calendar/Calendar_Enumerate.swift#L770
https://github.com/apple/swift-foundation/blob/main/Sources/FoundationEssentials/Calendar/Calendar_Enumerate.swift#L703C32-L703C46

[parkera]

Oh, and another thing. The API is of course all expressed as Int. So if we compress the storage in a way that prevents correctly round-tripping a value, we have to figure out what it means for someone to set a value which we can't store.

[benrimmington]

~960 bits

The optional fields each need an extra byte, and padding between them for alignment, so the total size will be approximately twice your estimate.

MemoryLayout<Int?>.size       //->  9 bytes
MemoryLayout<Int?>.stride     //-> 16 bytes
MemoryLayout<Int?>.alignment  //->  8 bytes

[hassila]

~960 bits

The optional fields each need an extra byte, and padding between them for alignment, so the total size will be approximately twice your estimate.

MemoryLayout<Int?>.size       //->  9 bytes
MemoryLayout<Int?>.stride     //-> 16 bytes
MemoryLayout<Int?>.alignment  //->  8 bytes

Yeah, thanks - it was just a rough estimate (thus the comment "(exluding the layout overhead for the optional enum, but just looking at data storage)"). Not just trivial replacement for the fields as noted by @parkera due to the support for adding values, but that gives us ~2K for a single instance, so should hopefully be some low hanging fruit here.

[parkera]

According to my measurements the current size/stride/alignment of DateComponents is:

266
272
8

[parkera]

We actually needed to treat Int.max as a nil value anyway for various compatibility reasons, so removing the optional storage and keeping a nil value as Int.max reduces the size to 160.

[hassila]

Just found this one: #206

[naknut]

Hi all! I just found this ticket and thought it could be a good first issue for me to look into. I see in @hassila's comment that there is a draft PR up for this. Is that a resonable implementation? As I understand it the PR is not merged because there needs to be tests added. Maybe I can pick that up and add the tests? That said, im a little unsure what additional tests needs to be added. Could @Azoy maybe help out and outline that?