Contribution Areas

[kamranayub]

I'm interested in contributing, as I'm a bit jaded with the current non-OSS pull server implementation. Specifically I'm interested in:

• Adding SQL server support (EF Core, pluggable provider, etc.)
• Adding proper reporting (OData or otherwise)
• Potentially exposing a reporting UI (I already have something I've built internally as a basic reporting UI)

The current storage (ESENT) and reporting is sub-optimal. The OData endpoint doesn't implement the spec fully and the ESENT provider doesn't even seem to work right (it might be fixed in 5.1). I would rather opt for a proper SQL server backend for reporting and metadata.

[concentrateddon]

Yeah, so, it's important to understand what Tug is, and what it isn't.

Tug is an ASP.NET Core web service that implements the pull server protocol. Tug doesn't have any logic, or any database, or anything. Instead, when a request is received from a node, Tug parses that request and then calls a predefined PowerShell command. You write that command. In that command, you can do whatever you want.

Tug, as a project, isn't ever going to have a data layer - it's just a middleman between nodes and your PowerShell commands.

So in terms of contribution areas, it'd be nice to get a very basic set of commands wired up as a starting point for people. Perhaps storing node data in text files for pure simplicity of illustration. You an then go on from there to create a much more complex set, if you desire.

[kamranayub]

I wonder if it might be useful to support both a Powershell command provider or a .NET Core provider. For example, it would be easier for my use case to implement a .NET core command handler(s) that used EF Core to store data in my database vs. having to use Powershell + ADO.NET. Thoughts?

[concentrateddon]

So, we should probably not use the word "provider" because it implies a level of framework that doesn't exist in the project. There's no provider; I personally don't have plans to create a provider layer. You're more than welcome to do so, but I'd argue that you could also just write your PowerShell commands in .NET (including Core) and achieve more or less the same effect (and, PowerShell itself has no hard requirement that you use ADO.NET). My goal with this project was to create a pull server that would be programmatically accessible to the many PowerShell scripters out there; you may not be that audience. And if that's the case, you're more than welcome to fork the base project so you've got the request-handling code, and make something else that's totally different and wonderful in its own right.

[kamranayub]

Thanks, that clarification is very helpful! I understand what you mean--i.e. just write it all in .NET, and use PS to invoke it like you would any other assembly.

[concentrateddon]

Yeah, and I should point out that my next step is going to be some load-testing with this model, because I need to make sure it's actually feasible. It might not be, in a cross-platform model, since .NET Core can't directly host PowerShell. So... I'll report back once I know more in that regard.

[ebekker]

Don, can you clarify what you mean by your last comment? Isn't it possible to host PS v6 (Core Edition) in a .NET Core host?

And if you code Tug against .NET Standard instead of specifically to .NET Core, then shouldn't it be possible to host even the Window-specific version (Desktop Edition)?

[concentrateddon]

No, it is not possible.

And yes, if I coded against Windows-only .NET, then it'd be possible to host the Windows Desktop version of the engine.

[ebekker]

So in that case, what is the strategy you're looking to use? Are you planning on doing some out-of-process approach to bridge the ASP.NET Core process with a separate PS Host?

[concentrateddon]

Well, you can see the strategy in the Runner.cs file ;). And yeah, I'm shelling out to run PowerShell itself.

[ebekker]

Ah, good ol' Process.Start() -- I did see that but I wasn't sure if it was just a placeholder for something more integrated.

I didn't quite understand your comment about not being able to host PS in Core until I did some searching around on the PowerShell repo -- so it looks like it's currently not possible but it is something in the works, so should be ok...eventually.

However, I was wondering if you considered inverting the model? So instead of a ASP.NET Core process hosting PowerShell -- maybe kick the service off as a PowerShell (v6) cmdlet which itself establishes the ASP.NET Core endpoint and route handling but hands back off to the PowerShell container.

I'm not exactly saying this is the best or proper way to handle it (or that it would even work) but just a thought...

[concentrateddon]

Yeah, performance on that is deeply suck. Tried it. It also doesn't work all that well functionally. Try it sometime.

[ebekker]

:-) I'm not surprised.

Looks like there might be a way to get this to work based on the comments here and here. I'm going to give it a try, and if it works, I'll submit a PR with an alternate Runner.

UPDATE: there is now some documented guidance on hosting PS6, so I've moved this goal over to #71.

[ebekker]

@concentrateddon -- I wanted to see if you would be open to supporting a provider model as @kamranayub had suggested for the purpose of furthering the project along before official support for PS6 embedding in a host is available?

Since ASP.NET Core is fully supported on but the .NET Framework, as well as .NET Core, it's possible to develop tug in such a way that it can support your design goals, but run atop PS5 initially, and then flipped over to PS6 once embedding is working.

A provider model would allow you to abstract that layer away and still develop the core protocol handling functionality atop ASP.NET Core MVC/Web API.

[concentrateddon]

Would you be willing to code it? You need to understand that my ASP.NET coding abilities are limited; being "open" to something doesn't necessarily give me the time or ability to create it. If, however, you'd like to contribute to the project, then fine.

[ebekker]

Oh absolutely -- I would be more than happy to contribute this piece of the project. Since you are the steward of this project, I wanted to make sure this is a direction you would be willing to take based on your earlier comments, but yes, I wasn't asking for you to go figure it out and add it in.

I have an OSS project that's enjoyed some modest success and one of the key features is its extensibility which is driven by a Provider-based extension mechanism. That project is based on the .NET Framework (PS5), but I've already begun work on a .NET Core (PS6) version with a ported and working Provider model that's based on Microsoft's MEF2.

It may be a bit overkill for this project at this stage, but I was thinking of setting up something along the same lines for tug.

[concentrateddon]

Works for me. I'm going to be doing some back-work to get this running under "normal" ASP.NET for now, but my goal is still to have it be Core-compatible. So go for it.

[thedevopsmachine]

Try replacing your "frameworks" entry in your project.json file with the following. It will allow you to extend your OS support to NanoServer.

"frameworks": {
"netstandard1.6": {
"dependencies": {
"Microsoft.PowerShell.NanoServer.NetCore.Library": "1.0.0"
}
},
"net452": {
"dependencies": {
"Microsoft.PowerShell.5.ReferenceAssemblies": "1.0.0"
}
}
}

[ebekker]

Thanks @thedevopsmachine -- I've opened #42 to track this as a separate activity to enable and test.

[kamranayub]

Hey team, I saw EFCore is in the works. Is there a discussion or planning topic around this? I'd be interested in maybe helping out there since Tug seems to be in a pretty good place now to start testing it out.

[ebekker]

Hi @kamranayub, the EFCore version is in the very early stages -- just conceptual at this point, so it's definitely a good time to start talking about it and any contributions are welcome. Go ahead and open a new issue and we'll use that to start planning and design discussions.

[CodeblackNL]

Hi,

As many, I'm looking for an alternative for the built-in DSC Pull Webservice. I would also like to contribute; most likely to the .NET Core & EF Core side of things, and make it run in a (Nano-based) container.

I would very much like a web frontend to be part of the DSC Pull Server. At least some sort of dashboard, that auto-updates with information from the registration, reporting & other activities from the nodes. That frontend does not need to be part of Tug itself, but Tug should provide a REST (and maybe SignalR) API for the frontend to consume. That API does not need to be built in, but could be added through some extensibility mechanism.

Are there any plans for supporting any kind of UI frontend?

[edthedev]

I'm pretty handy at building web APIs, but my organization isn't far enough down the road to DSC adoption for me to have a suitable test environment. If we do adopt DSC at any reasonable scale, and don't just jump strait to Azure, then helping add a REST API for Tug status is something I'm interested in helping with.

If I understand the Tug architecture correctly, the proposed status API will very likely not be built into Tug itself, but end up being a reference implementation of an integration with Tug that uses and updates a shared database.

[ebekker]

@CodeblackNL, @edthedev -- thanks for your interest, any contributions would be appreciated!

As far as UI front-end -- yes, I too have been thinking about this. As you say, it's not part of the core DSC bits of Tug, but there's no reason that it can't live side-by-side within the same host.

If you guys are interested in discussing/designing this piece some more, I suggest we pull this out into its own GH issue and continue the discussion there?

[ebekker]

@edthedev -- regarding adopting and testing, even if your org moves to Azure or any other environment, there is still great benefit in moving to DSC and adopting it in that environment, that's actually my own situation, and one of the reasons that I started the AWS Lambda version of Tug -- it's a serverless version of the DSC Pull Service that can service any node (even Linux) in the environment. And I'm going to be kicking off the Azure version of this (Azure Functions) shortly.

[ebekker]

All, I'm closing this ticket out -- if you have any specific suggestions or requests, please open individual issues.