Skip to content

Integration of Specs2 with the Scala IDE

Jump to bottom Edit New page
rlegendi edited this page Aug 30, 2012 · 25 revisions

Integration of Specs2 with the Scala IDE

1   Abstract

I would be interested in joining the development of the Scala IDE. This project is interesting for me because software testing is a field that I find extremely important. I believe making it easier for developers to create, run, and verify the results of such tests and executable specifications is an essential feature. My goal is to contribute a new plugin which can easily manage tests created by the Specs2 framework, and possibly a simple tutorial about its usage. I believe implementing such an easy-to-use testing feature to the Scala IDE may really help developers and make the environment a more widely used tool.

  • Project Ideas (See section Integration of specs2 with Scala IDE for Eclipse)
  • Original proposal (I have just made it public, by default only the abstract was shown)

2   Rough Plan

2.1   Before the actual coding period starts (until 5/21/2012)

  • DONE Contact Eric and Mirco about the project
  • DONE Subscribe for the scala-lang, specs2, scala-ide, scala-ide-dev mailing lists
  • DONE Read a few Scala books (most important for the moment is the Predef object, sys package and the collections)
  • DONE Do the Startup guide, make the developer environment ready
  • DONE Read the Workflow guide
    • DONE Sign the CLA for the Scala-IDE team
  • DONE Read the documentation of Specs2
    • Have a Skype meeting with Eric to get know the internals
  • DONE Watch a few videos about Scala/Specs2
    • Eric's talk on Specs2
    • Check out the ScalaDays 2012 videos
    • Especially Bill Venners' talk about ScalaTest integration with the Scala IDE. Mirco's idea: "Furthermore, Bill's ScalaTest plugin should also be able to run Specs2 tests (if I got this right, it is enough to either use an annotation or mix-in a special trait). I believe there may be some overlap between ScalaTest and Specs2 integration, so I would suggest you guys to get in touch with Bill and discuss how you can avoid duplication of efforts. This would leave you more time for developing Specs2 specific functionalities, I'm sure Eric has many ideas ;)" Note: I have located the relevant pull request, I should write a note to Chua Chee Seng who made the pull request and examine the code.
  • DONE Check what other tools can do with Specs2 like ScalaTest integration (as written before), or IntelliJ (here's the code thanks to Eric), and how other tests run

2.2   Until the Mid-term Evaluation

What I believe would be useful is to have a rich set of features (like templates, wizards for creating different tests, context-sensitive commands to run them, run all of them separately or in a batch, etc.).

  • DONE Create a separate plugin (like org.scala-ide.sdt.debug)
  • DONE Create a minimal Github-page with a few screensots
  • DONE Create a prototype
  • DONE Write tests in Specs2 to the new features (should be run in headless mode) to dog-feed the plugin
  • DONE Templates It would be nice to have some templates like must, should, Specification, etc. (Examples included in the scala-ide\org.scala-ide.sdt.core\templates\default-templates.xml file)

2.3   Until the Final Evaluation

Ideas of possible new features (these having priorities at the moment):

  • [DONE] Proper error display For the failing tests, there must be a way to properly show the reason for the user.
  • [DONE] Support Specs2 Data Tables See Specs2 documentation Probably by using code snippets?
  • [PENDING] Formatting tools Reformatting data tables and specifications (^ characters to the end of line) would be a rocking feature (check out how Scalariform is used within the Scala IDE).
  • [PENDING] Make the framework find SpecificationStructures without the @WrapWith annotation Probably runner.SpecificationsFinder could help us.

Other developments:

  • Adapt to the Scala-IDE API (Pending)
    • As soon as we have a fork, Mirco can set up a nightly build on their CI server
    • Crete a separate Github project instead of developing in Scala-IDE project (pending)
    • Requires a new feature definition, update site, etc.
  • Write a short introductory tutorial (or videocast) on the usage of the plug-in
  • Support Specs Forms See Specs2 documentation Probably by using code snippets?
  • Additional syntax-highlight for Specs2-specific functions like should, can, in, etc.
  • Welcome screen components I saw that there is no welcome screen addition for the Scala IDE currently, so it might be interesting for them too if we would have a Specs2 entry there (probably links to tutorials and the project website at Github).
  • Integration with the Scala IDE API This is a pending issue, since the API is currently under development.
  • Exporting features Document generations (e.g., HTML reports)
  • Code coverage generation Scct?
  • Check supported platforms Eclipse Juno, different Scala versions (2.8, 2.9, 2.10)

3   Pending issues

  • Spec-in-spec issue: order of notification calls
  • Modifiers/doc gen action for the new wizards
  • Error locations needs to be consulted with Eric

3.1   Prototyping with ScalaTest

cheeseng and bvenners gave me a quick reply for the idea of using ScalaTest as a platform for our project to integrate Specs2 into the Scala IDE. Eric also suggested this way of development. Specs1Runner could be easily used to bridge the two worlds together.

3.2   Adding The Sepcs2 Nature and Libraries to the Project

I imagine adding the required libraries to a given Scala project a similar way how JUnit 3/4 libraries are added to a general Java project in Eclipse:

  1. Right-click on the project, select Build Path --> Configure Build Path...
  2. Click on Add Library
  3. There will be a Specs2 option in the wizard

3.3   Installing the Plugin

(This is relevant once we have at least a set of basic functionalities and the project is moved to a separate Github project.)

There will be a separate feature for the project, allowing us to create an update site for the included plugins. This way the new functionalities might be installed into the Scala IDE by the standart Eclipse way under the Help --> Install new software... item by adding the new site to the existing ones. Also, updating the plugin will be easier.

3.4   Running the Tests

There should be a new RunConfiguration and RunConfigurationTabGroup in the relevant wizard in Eclipse at the run configurations. We need proper RunAction implementations.

A new run configuration should be automatically created when the user start the tests via the Package Explorer (assuming there was na such proper configuration before), right-clicking on the either on a source folder, test class or either an expanded test case from the list of a class, and selecting a Run As... --> Spec2 specification.

The output of the test should be directed to a new, specified Console (this is required for some additional features like red error messages and blue/black standard text the test might output, and to allow further customization).

3.5   Results of the Test

The result would be shown in a separate View next to the Project Explorer (just like when JUnit tests are run in Eclipse).

Idea: running the tests might be based on the current architecture Eclipse has around JUnit. Some parts of the code might be usable in this project (e.g., the visualization), and the functionalities might be similiar too (e.g., jump to a specific test/line, show the output of a failed assertion, etc.). Probably a fast prototyping in Java could validate this approach.

Planned features on the new view:

  • Display the number of sussessful/ignored/failed tests.
  • Display the progress during running the tests.
  • On failed specifications, show the failure trace.
  • Option to terminate the current test runs.
  • (Optional) Filter results to failed tests only.
  • (Optional) Navigate between failed test traces.
  • (Optional) Rerun the selected set of tests.
  • (Optional) Rerun the selected set of tests, starting with the ones that failed previously.
  • (Optional) Utilize Eclipse's Test Run History facility.

4   Other Notes

  • Submit a presentation proposal for the local Eclipse DemoCamp (I had a few presentations so far)
  • Summer school: There will be a Summer School on programming languages at my university where Scala has a half-day session. I might be able to promote Scala-IDE and Specs2 there :-)
  • Book chapter on parallel language constructs: I will participate in the extension and English translation of the book Programming Languages. Probably I will be able to promote Scala/Scala IDE/Specs2 :-)

5   Future Work

If I manage to finish the project sooner, we might add some additional features awaiting to be implemented in the Scala IDE, like:

  • Some basic CompletionProposals or ContentAssistProcessors for tests.
  • Custom TextHovers.
  • Specific local menu items for the editor of tests (e.g., Ignore/"Unignore" tests through some EditorActionDelegates for instance), or run selected tests from the current Editor (if applicable).
  • Place (Error)Markers to places where the tests failed.
  • ShortcutConfigurations to running the tests (e.g., Ctrl X + T, S, to follow Eclipse's built-in mechanism).
  • Some ConfigurationTab/PreferencePage if we find something relevant setting globally for the tests.
  • Customized Outline for tests.
  • Customized Perspective for tests.

Unfortunately, I am just getting to know the Scala IDE in details, so some (or all) of my previous ideas might already be implemented.

Add a custom footer
Add a custom sidebar
Clone this wiki locally