Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync ReadLimiter #201

Merged
merged 4 commits into from
Dec 19, 2020
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion capnp/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ quickcheck = { version = "0.9", optional = true }
quickcheck = "0.9"

[features]
default = ["std"]
default = ["std", "sync_reader"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to make the new feature not a default feature, so that we minimize the chance of breaking anyone's code.


rpc_try = []

Expand All @@ -37,3 +37,8 @@ unaligned = []
# with the Rust standard library.
std = []

# If disabled, message reader will not be `Sync`. This can be used
# to support platforms where `AtomicUsize` is not available.
# To be replaced by #[cfg(target_has_atomic = ...)] once it lands.
# See: https://github.com/rust-lang/rust/issues/32976
sync_reader = []
28 changes: 4 additions & 24 deletions capnp/src/private/arena.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,38 +19,18 @@
// THE SOFTWARE.

use alloc::vec::Vec;
use core::cell::{Cell, RefCell};
use core::cell::RefCell;
use core::slice;
use core::u64;

use crate::private::units::*;
use crate::private::read_limiter::ReadLimiter;
use crate::message;
use crate::message::{Allocator, ReaderSegments};
use crate::{Error, OutputSegments, Result};

pub type SegmentId = u32;

pub struct ReadLimiter {
pub limit: Cell<u64>,
}

impl ReadLimiter {
pub fn new(limit: u64) -> ReadLimiter {
ReadLimiter { limit: Cell::new(limit) }
}

#[inline]
pub fn can_read(&self, amount: u64) -> Result<()> {
let current = self.limit.get();
if amount > current {
Err(Error::failed(format!("read limit exceeded")))
} else {
self.limit.set(current - amount);
Ok(())
}
}
}

pub trait ReaderArena {
// return pointer to start of segment, and number of words in that segment
fn get_segment(&self, id: u32) -> Result<(*const u8, u32)>;
Expand Down Expand Up @@ -129,12 +109,12 @@ impl <S> ReaderArena for ReaderArenaImpl<S> where S: ReaderSegments {
if !(start >= this_start && start - this_start + size <= this_size) {
Err(Error::failed(format!("message contained out-of-bounds pointer")))
} else {
self.read_limiter.can_read(size_in_words as u64)
self.read_limiter.can_read(size_in_words)
}
}

fn amplified_read(&self, virtual_amount: u64) -> Result<()> {
self.read_limiter.can_read(virtual_amount)
self.read_limiter.can_read(virtual_amount as usize)
}
}

Expand Down
1 change: 1 addition & 0 deletions capnp/src/private/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ mod primitive;
pub mod layout;
mod mask;
pub mod units;
mod read_limiter;
mod zero;

#[cfg(test)]
Expand Down
92 changes: 92 additions & 0 deletions capnp/src/private/read_limiter.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
// Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
// Licensed under the MIT License:
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.

#[cfg(feature = "sync_reader")]
pub use sync::ReadLimiter;

#[cfg(feature = "sync_reader")]
mod sync {
use crate::{Error, Result};
use core::sync::atomic::{AtomicUsize, Ordering};

pub struct ReadLimiter {
pub limit: usize,
pub read: AtomicUsize,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making this an AtomicU64 would more closely match the current implementation, which uses a u64. I'm worried that someone might be setting the current limit to a large u64 value, and when this new feature gets enabled their code will break.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue with AtomicU64, though, is that it is supported on fewer platforms. I don't see a nice solution here...

}

impl ReadLimiter {
pub fn new(limit: u64) -> ReadLimiter {
if limit > core::usize::MAX as u64 {
panic!("traversal_limit_in_words cannot be bigger than core::usize::MAX")
}

ReadLimiter {
limit: limit as usize,
read: AtomicUsize::new(0),
}
}

#[inline]
pub fn can_read(&self, amount: usize) -> Result<()> {
let read = self.read.load(Ordering::Relaxed) + amount;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks to me like this can overflow if self.read.load(Ordering::Relaxed) + amount > core::usize::MAX.

Why not write this is the same way as the non-Sync version, with a single limit: AtomicUsize field that decreases on each call? I think overflows and underflows are easier to avoid in that case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, unless I don't see an obvious solution here, the way it's done in the non-sync version is possible because it's single threaded. In a multi-threaded version, I need to make sure that the limit underflowing is handled correctly. If I was to get then sub without checking underflow, the limit would wrap around and allows the next readers to read usize::MAX.

I think the version I pushed is better and handle this better than my previous version. I use an extra flag to indicate that the limit has been reached when the limit got to 0 or when it underflows.


if read > self.limit {
Err(Error::failed(format!("read limit exceeded")))
} else {
self.read.fetch_add(amount, Ordering::Relaxed);
Ok(())
}
}
}
}

#[cfg(not(feature = "sync_reader"))]
pub use unsync::ReadLimiter;

#[cfg(not(feature = "sync_reader"))]
mod unsync {
use core::cell::Cell;
use crate::{Error, Result};

pub struct ReadLimiter {
pub limit: Cell<u64>,
}

impl ReadLimiter {
pub fn new(limit: u64) -> ReadLimiter {
ReadLimiter {
limit: Cell::new(limit),
}
}

#[inline]
pub fn can_read(&self, amount: usize) -> Result<()> {
let amount = amount as u64;
let current = self.limit.get();
if amount > current {
Err(Error::failed(format!("read limit exceeded")))
} else {
self.limit.set(current - amount);
Ok(())
}
}
}
}