Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make (some) Ice properties opt-in (C++ only) #2983

Merged
merged 15 commits into from
Oct 30, 2024
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,8 +118,9 @@ jobs:
if-no-files-found: ignore
if: always()

# - name: Test Summary
# uses: test-summary/action@v2
# with:
# paths: "${{ matrix.working_directory || '.' }}/test-report.xml"
# if: always()
- name: Upload macOS crash diagnostics
uses: actions/upload-artifact@v4
with:
name: crash-diagnostics-${{ matrix.config }}-${{ matrix.os }}
path: ~/Library/Logs/DiagnosticReports/*.ips
if: runner.os == 'macOS' && always()
30 changes: 15 additions & 15 deletions config/PropertyNames.xml
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@
<property name="Reject.[any]" languages="cpp,csharp,java" />
</class> -->

<section name="Ice">
<section name="Ice" service-prefix="false">
<property name="AcceptClassCycles" languages="cpp,csharp,java" default="0" />
<property name="Admin" class="ObjectAdapter" languages="cpp,csharp,java" />
<property name="Admin.DelayCreation" languages="cpp,csharp,java" default="0" />
Expand Down Expand Up @@ -160,7 +160,7 @@
<property name="ThreadInterruptSafe" languages="java" />
</section>

<section name="IceMX">
<section name="IceMX" service-prefix="false">
<property name="Metrics.[any].GroupBy" languages="cpp,csharp,java" />
<property name="Metrics.[any].Map" languages="cpp,csharp,java" />
<property name="Metrics.[any].RetainDetached" default="10" languages="cpp,csharp,java" />
Expand All @@ -169,7 +169,7 @@
<property name="Metrics.[any]" languages="cpp,csharp,java" />
</section>

<section name="IceDiscovery">
<section name="IceDiscovery" service-prefix="false">
<property name="Multicast" class="ObjectAdapter" languages="cpp,csharp,java" />
<property name="Reply" class="ObjectAdapter" languages="cpp,csharp,java"/>
<property name="Locator" class="ObjectAdapter" languages="cpp,csharp,java" />
Expand All @@ -184,7 +184,7 @@
<property name="DomainId" languages="cpp,csharp,java" default=""/>
</section>

<section name="IceLocatorDiscovery">
<section name="IceLocatorDiscovery" service-prefix="false">
<property name="Reply" class="ObjectAdapter" languages="cpp,csharp,java" />
<property name="Locator" class="ObjectAdapter" languages="cpp,csharp,java" />
<property name="Lookup" languages="cpp,csharp,java"/>
Expand All @@ -198,7 +198,7 @@
<property name="Trace.Lookup" languages="cpp,csharp,java" default="0" />
</section>

<section name="IceBox">
<section name="IceBox" service-prefix="true">
<property name="InheritProperties" languages="cpp,csharp,java" />
<property name="LoadOrder" languages="cpp,csharp,java" />
<property name="PrintServicesReady" languages="cpp,csharp,java" />
Expand All @@ -207,17 +207,17 @@
<property name="UseSharedCommunicator.[any]" languages="cpp,csharp,java" />
</section>

<section name="IceBoxAdmin">
<section name="IceBoxAdmin" service-prefix="true">
<property name="ServiceManager.Proxy" class="Proxy" languages="cpp,java" />
</section>

<section name="IceBridge">
<section name="IceBridge" service-prefix="true">
<property name="Source" class="ObjectAdapter" languages="cpp" />
<property name="Target.Endpoints" languages="cpp" />
<property name="InstanceName" default="IceBridge" languages="cpp" />
</section>

<section name="IceGridAdmin">
<section name="IceGridAdmin" service-prefix="true">
<property name="AuthenticateUsingSSL" languages="cpp" />
<property name="MetricsConfig" languages="java" />
<property name="Username" languages="cpp" />
Expand All @@ -236,7 +236,7 @@
<property name="Trace.SaveToRegistry" languages="java" />
</section>

<section name="IceGrid">
<section name="IceGrid" service-prefix="true">
<property name="AdminRouter" class="ObjectAdapter" languages="cpp" />
<property name="InstanceName" languages="cpp" default="IceGrid" />
<property name="Node" class="ObjectAdapter" languages="cpp" />
Expand Down Expand Up @@ -301,7 +301,7 @@
<property name="Registry.UserAccounts" languages="cpp" />
</section>

<section name="IceSSL">
<section name="IceSSL" service-prefix="false">
<property name="Alias" languages="java" />
<property name="CAs" languages="cpp,csharp"/>
<property name="CertStore" languages="cpp,csharp" default="My" />
Expand Down Expand Up @@ -333,7 +333,7 @@
<property name="VerifyPeer" languages="cpp,csharp,java" default="2" />
</section>

<section name="IceStorm">
<section name="IceStorm" service-prefix="true">
<property name="Discard.Interval" languages="cpp" default="60" />
<property name="Election.ElectionTimeout" languages="cpp" default="10" />
<property name="Election.MasterTimeout" languages="cpp" default="10" />
Expand All @@ -360,18 +360,18 @@
<property name="Transient" languages="cpp" default="0" />
</section>

<section name="IceStormAdmin">
<section name="IceStormAdmin" service-prefix="true">
<property name="TopicManager.[any]" languages="cpp" />
<property name="Host" languages="cpp" />
<property name="Port" languages="cpp" />
</section>

<section name="IceBT">
<section name="IceBT" service-prefix="false">
<property name="RcvSize" languages="cpp,java" />
<property name="SndSize" languages="cpp,java" />
</section>

<section name="Glacier2">
<section name="Glacier2" service-prefix="true">
<property name="AddConnectionContext" languages="cpp" default="0" />
<property name="Client" languages="cpp" class="ObjectAdapter"/>
<property name="Client.ForwardContext" languages="cpp" default="0" />
Expand All @@ -398,7 +398,7 @@
<property name="Trace.Session" languages="cpp" default="0" />
</section>

<section name="DataStorm">
<section name="DataStorm" service-prefix="true">
<property name="Node.ConnectTo" languages="cpp" />
<property name="Node.Multicast" class="ObjectAdapter" languages="cpp" />
<property name="Node.Multicast.Enabled" default="1" languages="cpp" />
Expand Down
42 changes: 29 additions & 13 deletions config/makeprops.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,12 @@ class Language(StrEnum):


class PropertyArray:
def __init__(self, name: str, prefixOnly: bool, isClass: bool):
def __init__(
self, name: str, prefixOnly: bool, isClass: bool, isServicePrefix: bool
):
self.name = name
self.prefixOnly = prefixOnly
self.isServicePrefix = isServicePrefix
self.isClass = isClass
self.properties = []

Expand Down Expand Up @@ -180,17 +183,28 @@ def startElement(self, name, attrs):
case "properties":
pass
case "class":
self.validateKnownAttributes(["name", "prefix-only"], attrs)
self.parentNodeName = f"{attrs.get("name")}"
name = f"{attrs.get("name")}"
prefixOnly = attrs.get("prefix-only", "false").lower() == "true"
self.propertyArrayDict[self.parentNodeName] = PropertyArray(
self.parentNodeName, prefixOnly, True

self.validateKnownAttributes(["name", "prefix-only"], attrs)
self.parentNodeName = name
self.propertyArrayDict[name] = PropertyArray(
name=name,
prefixOnly=prefixOnly,
isClass=True,
isServicePrefix=False,
)
case "section":
self.validateKnownAttributes(["name"], attrs)
self.parentNodeName = attrs.get("name")
isServicePrefix = attrs.get("service-prefix", "false").lower() == "true"
name = attrs.get("name")

self.validateKnownAttributes(["name", "service-prefix"], attrs)
self.parentNodeName = name
self.propertyArrayDict[self.parentNodeName] = PropertyArray(
self.parentNodeName, False, False
name=name,
prefixOnly=False,
isClass=False,
isServicePrefix=isServicePrefix,
)

case "property":
Expand Down Expand Up @@ -263,6 +277,7 @@ def openFiles(self):
const bool prefixOnly;
const Property* properties;
const int length;
const bool isServicePrefix;
}};

class PropertyNames
Expand Down Expand Up @@ -304,7 +319,7 @@ def fix(self, propertyName):
def writePropertyArray(self, propertyArray):
name = propertyArray.name
prefixOnly = "true" if propertyArray.prefixOnly else "false"

isServicePrefix = "true" if propertyArray.isServicePrefix else "false"
self.hFile.write(f" static const PropertyArray {name}Props;\n")

self.cppFile.write(f"""\
Expand All @@ -315,10 +330,11 @@ def writePropertyArray(self, propertyArray):

const PropertyArray PropertyNames::{name}Props
{{
"{name}",
{prefixOnly},
{name}PropsData,
{len(propertyArray.properties)}
.name="{name}",
.prefixOnly={prefixOnly},
.properties={name}PropsData,
.length={len(propertyArray.properties)},
.isServicePrefix={isServicePrefix}
}};

""")
Expand Down
103 changes: 87 additions & 16 deletions cpp/include/DataStorm/Node.h
Original file line number Diff line number Diff line change
Expand Up @@ -38,47 +38,115 @@ namespace DataStorm
/**
* Construct a DataStorm node.
*
* A node is the main DataStorm object. It is required to construct topics. The constructor initializes
* the Ice communicator using the given arguments. If the communicator creation fails, an Ice exception is
* raised.
* A node is the main DataStorm object. It is required to construct topics. The node uses the given Ice
Copy link
Member Author

@externl externl Oct 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We needed to be able to intercept the construction of the Node's communicator to create a Property set with the "DataStorm" service prefix. So we can no longer just forward everything. @bernardnormier suggested we just offer a few constructors. This also means we can now always have the custom executor as a trailing parameter.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The alternative is to not consider DataStorm a service, which is not really one. It is a user library given that you have to programmatically construct the node.

In theory any servers communicator can be shared with the node and pass to its constructor.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer making DataStorm an opt-in reserved prefix, as opposed to allowing it all the time like the plugins.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this the same as IceDiscovery and IceLocatorDiscovery which are marked with service-prefix=false.?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the plugins (IceDiscovery etc.) are treated like Ice - always enabled.

* communicator.
*
* @param argc The number of arguments in argv.
* @param argv The configuration arguments.
* @param configFile The path to an optional Ice configuration file.
* @param customExecutor An optional executor used to execute user callbacks, if no callback executor is
* provided the Node will use the default callback executor that executes callback in a dedicated thread.
*
* @param iceArgs Arguments which are passed to the Ice::initialize function.
*/
template<class... T> Node(T&&... iceArgs) : _ownsCommunicator(true)
Node(
int& argc,
const char* argv[],
std::optional<std::string_view> configFile = std::nullopt,
std::function<void(std::function<void()> call)> customExecutor = nullptr);

/**
* Construct a DataStorm node.
*
* A node is the main DataStorm object. It is required to construct topics. The node uses the given Ice
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This constructor (like most Node constructors) does not accept a communicator. So:

The node uses the given Ice communicator.

is not correct.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

* communicator.
*
* @param argc The number of arguments in argv.
* @param argv The configuration arguments.
* @param configFile The path to an optional Ice configuration file.
* @param customExecutor An optional executor used to execute user callbacks, if no callback executor is
* provided the Node will use the default callback executor that executes callback in a dedicated thread.
*
*/
Node(
int& argc,
char* argv[],
std::optional<std::string_view> configFile = std::nullopt,
std::function<void(std::function<void()> call)> customExecutor = nullptr)
: Node(argc, const_cast<const char**>(argv), configFile, customExecutor)
{
init(Ice::initialize(std::forward<T>(iceArgs)...), nullptr);
}

#ifdef _WIN32
/**
* Construct a DataStorm node.
*
* A node is the main DataStorm object. It is required to construct topics. The node uses the given Ice
* communicator.
*
* @param communicator The Ice communicator used by the topic factory for its configuration and communications.
* @param argc The number of arguments in argv.
* @param argv The configuration arguments.
* @param configFile The path to an optional Ice configuration file.
* @param customExecutor An optional executor used to execute user callbacks, if no callback executor is
* provided the Node will use the default callback executor that executes callback in a dedicated thread.
*
*/
Node(
Ice::CommunicatorPtr communicator,
int& argc,
const wchar_t* argv[],
std::optional<std::string_view> configFile = std::nullopt,
std::function<void(std::function<void()> call)> customExecutor = nullptr);

/**
* Construct a DataStorm node.
*
* A node is the main DataStorm object. It is required to construct topics. The constructor initializes
* the Ice communicator using the given arguments. If the communicator creation fails, an Ice exception is
* raised.
* A node is the main DataStorm object. It is required to construct topics. The node uses the given Ice
* communicator.
*
* @param argc The number of arguments in argv.
* @param argv The configuration arguments.
* @param configFile The path to an optional Ice configuration file.
* @param customExecutor An optional executor used to execute user callbacks, if no callback executor is
* provided the Node will use the default callback executor that executes callback in a dedicated thread.
* @param iceArgs Arguments which are passed to the Ice::initialize function.
*
*/
template<class... T>
Node(std::function<void(std::function<void()> call)> customExecutor, T&&... iceArgs) : _ownsCommunicator(true)
Node(
int& argc,
wchar_t* argv[],
std::optional<std::string_view> configFile = std::nullopt,
std::function<void(std::function<void()> call)> customExecutor = nullptr)
: Node(argc, const_cast<const wchar_t**>(argv), configFile, customExecutor)
{
init(Ice::initialize(std::forward<T>(iceArgs)...), std::move(customExecutor));
}
#endif

/**
* Construct a DataStorm node.
*
* A node is the main DataStorm object. It is required to construct topics. The node uses the given Ice
* communicator.
*
* @param configFile The path to an optional Ice configuration file.
* @param customExecutor An optional executor used to execute user callbacks, if no callback executor is
* provided the Node will use the default callback executor that executes callback in a dedicated thread.
*/
Node(
std::optional<std::string_view> configFile = std::nullopt,
std::function<void(std::function<void()> call)> customExecutor = nullptr);

/**
* Construct a DataStorm node.
*
* A node is the main DataStorm object. It is required to construct topics. The node uses the given Ice
* communicator.
*
* @param communicator The Ice communicator used by the topic factory for its configuration and communications.
* This communicator must be initialized with a property set to use the "DataStorm" service prefix.
* @param customExecutor An optional executor used to execute user callbacks, if no callback executor is
* provided the Node will use the default callback executor that executes callback in a dedicated thread.
*/
Node(
Ice::CommunicatorPtr communicator,
std::function<void(std::function<void()> call)> customExecutor = nullptr);

/**
* Construct a new Node by taking ownership of the given node.
Expand Down Expand Up @@ -133,7 +201,10 @@ namespace DataStorm
Ice::ConnectionPtr getSessionConnection(const std::string& ident) const noexcept;

private:
void init(const Ice::CommunicatorPtr&, std::function<void(std::function<void()> call)> customExecutor);
Node(
const Ice::CommunicatorPtr&,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be an Ice::CommunicatorPtr.

std::function<void(std::function<void()> call)> customExecutor,
bool ownsCommunicator);

std::shared_ptr<DataStormI::Instance> _instance;
std::shared_ptr<DataStormI::TopicFactory> _factory;
Expand Down
5 changes: 3 additions & 2 deletions cpp/include/Ice/LocalExceptions.h
Original file line number Diff line number Diff line change
Expand Up @@ -934,9 +934,10 @@ namespace Ice
};

/**
* An unknown property was used when trying to get or set an unknown property.
* This exception is raised when there is an error while getting or setting a property. For example, when
* trying to set an unknown Ice property.
*/
class ICE_API UnknownPropertyException final : public LocalException
class ICE_API PropertyException final : public LocalException
{
public:
using LocalException::LocalException;
Expand Down
Loading
Loading