Update First-Time Build Tutorial

[Bckempa]

The first time build tutorial is out of date and still refers to the initial Earthly prototype branch. This needs to be updated to match the USAGE.md in the space-ros repo.

This has been noticed before, most recently by @Mechazo11 in space-ros/space-ros#187

[roboPanda69]

Could I take this issue and update the first time build tutorial page ?

[Bckempa]

Please do! We're always happy to welcome new contributors. Open a PR mentioning this issue when you're ready and we'll find a reviewer.

[roboPanda69]

Hi Bckempa, I tried if we could follow the same method, which is given in the first time build tutorial page. For the task 2, i.e., Bootstrap the Earthly configuration file, I wanted to add few lines in the Earthfile of space-ros -

earthfile:
  COPY Earthfile Earthfile
  COPY excluded-pkgs.txt ./
  COPY docker/ docker/
  COPY ros2.repos ./

  SAVE ARTIFACT Earthfile
  SAVE ARTIFACT excluded-pkgs.txt
  SAVE ARTIFACT docker/
  SAVE ARTIFACT ros2.repos

I wanted to add this line, so that when we use earthly --artifact github.com/space-ros/space-ros:+earthfile/Earthfile we are able to get the Earthfile, and the excluded-pkgs.txt, ros2.repos and docker file are important too for building +sources target and +build-testing target [Else we get error that these files are not provided]. I wanted to ask if I could create a pull request in space-ros directory too for changes in the Earthfile ?

[Bckempa]

@xfiderek, should the instructions still use the earthly --artifact method like the old docs, or should everyone be instructed to clone and build the way build.sh does in USAGE.md?

[Mechazo11]

@Bckempa thanks for mentioning my first-time build issue that we looked into a few weeks prior. Since then, I have been experimenting with the mars rover code and I ended up figuring out a way to install space-ros natively. The reason I did was the Mars rover demo was running really slow from Docker and was making it very difficult to observe the rover's behavior.

Sir why no native build instruction was given in the first-time build? Would such instruction be beneficial or is it preferable to use docker-style development for space-ros?

[xfiderek]

@xfiderek, should the instructions still use the earthly --artifact method like the old docs, or should everyone be instructed to clone and build the way build.sh does in USAGE.md?

@Bckempa Ok, now I understand what was the use case for this target. I believe we removed that after migration from docker repo, since nobody knew what was the purpose of it.

Long story short, we should update docs to use ./build.sh now, but would be good to keep the description what this file does in the docs, since majority of users will be new to earthly.

[xfiderek]

@Bckempa thanks for mentioning my first-time build issue that we looked into a few weeks prior. Since then, I have been experimenting with the mars rover code and I ended up figuring out a way to install space-ros natively. The reason I did was the Mars rover demo was running really slow from Docker and was making it very difficult to observe the rover's behavior.

Sir why no native build instruction was given in the first-time build? Would such instruction be beneficial or is it preferable to use docker-style development for space-ros?

@Mechazo11 to your point of installing spaceros natively - I think it should be quite straightforward (assuming you are building on x86). Try the following:

# clone source files and copy the `src` directory to the host machine
earthly +sources 
# build spaceros and copy the `install` directory to the host machine
earthly +build 
source install/setup.bash
# install all system dependencies
rosdep install --from-path src --ignore-src --skip-keys "$(tr '\n' ' ' < 'excluded-pkgs.txt') urdfdom_headers ikos" -y

The last command is taken from Earthfile
Let me know if it works. If so, we can add it to the docs.

[Mechazo11]

@xfiderek, the above commands worked though I think it took quite a while for the workspace to build (15 mins +) on a corei 15 9th gen Laptop with 16Gb ram. Is there a way we can speed it up. I think when the files were being built, earthly may not have been using more than one or two cores.

+build | Summary: 212 packages finished [15min 48s]
              +build |   7 packages had stderr output: cobra_vendor cyclonedds google_benchmark_vendor iceoryx_posh libyaml_vendor mimick_vendor rclpy
 +spaceros-artifacts | *cached* --> SAVE ARTIFACT ros2.repos +spaceros-artifacts/ros2.repos
 +spaceros-artifacts | *cached* --> SAVE ARTIFACT exact.repos +spaceros-artifacts/exact.repos
 +spaceros-artifacts | *cached* --> SAVE ARTIFACT excluded-pkgs.txt +spaceros-artifacts/excluded-pkgs.txt
              +build | --> COPY +spaceros-artifacts/exact.repos install/exact.repos
              +build | --> SAVE ARTIFACT install +build/install AS LOCAL install
              output | --> exporting outputs
              output | sent 1 file stat
              output | [----------] 100% exporting outputs

[Bckempa]

Sir why no native build instruction was given in the first-time build? Would such instruction be beneficial or is it preferable to use docker-style development for space-ros?

In general the focus on dockerized development in the early stages of the project was to reduce the number of platform configurations that needed to be supported, which can be especially valuable when working across organizations where ever developer could be constrained by different IT and security policies preventing them from selecting a consensus configuration.

There is no reason to exclude native build instructions from the docs, in fact having them for platform integrators is helpful - but I'd suggest that it might be simpler to leave the first-time walkthrough on docker so you don't need as many "depending on your system" statements to get people started.

You also have to keep in mind that Space ROS isn't just for ROS devs building for space, but also for flight software devs trying to support more roboticists so the whole ROS meta-(meta(meta-))build procedure won't be as familiar to that segment of users. That's another reason the docker "here's a full dev environment in a box" approach was used.

[Bckempa]

Long story short, we should update docs to use ./build.sh now, but would be good to keep the description what this file does in the docs, since majority of users will be new to earthly.

@roboPanda69 I think this answers your question if you are still interested in taking this on.

[Bckempa]

The last command is taken from Earthfile Let me know if it works. If so, we can add it to the docs.

I'll give this a try when I get back to my riscV Space ROS trials 😉

[Bckempa]

@xfiderek I will try it and report back. I took an overly complicated approach that involved using vcs, colon and rosdep. But this command would have made the process very simple. Would it be possible we make another earth file that builds the space-ros, moveit2 and space_robot as three separate workspaces so that the demos can be run natively?

There is on-going work to update the stack and containers: space-ros/space-ros#107
The design discussion is hosted here: space-ros/space-ros#140
More discussion about the design decisions that lead to the current build system, including a summery, are here: space-ros/space-ros#189
Can you elaborate on your thoughts about the sacks in the design discussion? We're all way off-topic from the first time docs on this issue and I don't want these thoughts to get lost.

@xfiderek awesome, these commands worked though I think it took quite a while for the workspace to build (15 mins +) on a corei 15 9th gen Laptop with 16Gb ram. Is there a way we can speed it up. I think when the files were being built, earthly may not have been using more than one or two cores.

When I build on my desktop there are times it pins all the cores I have but we already have three issues tracking known bottlenecks in the process: space-ros/space-ros#103, space-ros/space-ros#181, and space-ros/docker#145. If you find others or would like to contribute to addressing these we'd all be happy to see the build times improve. That said, build speed isn't the highest priority because most developers using Space ROS should be doing their own work on top of a prebuilt image instead of rebuilding the whole distro.

[Mechazo11]

@Bckempa thank you very much for the detailed discussion. I shall elaborate more on my thoughts about using earthly files for a native build of space_robots image later in design discussion early next week.

[damianphim]

I didn't see that someone already claimed this issue before I started working on it. I'm currently working on the first-time build as well as the canadarm and mars rover demo docs, is that okay?

[Bckempa]

This is actually a duplicate of the older #29 issue, however they both have attached PRs now... we'll sort this out after review.