Questions about the development environment

[wimjongman]

This thread is locked because there were too many separate discussions. Please open a new topic with your question.

[wimjongman]

@wimjongman Which "application" corresponds to the BIRT designer (= what happens when I execute the eclipse binary in above package)? I just assumed that it's the default (DesignerApplication) and then I would assume that it behaves in the same way

@

@wimjongman Which "application" corresponds to the BIRT designer (= what happens when I execute the eclipse binary in above package)? I just assumed that it's the default (DesignerApplication) and then I would assume that it behaves in the same way

@patric-r
You have to take a look at the available applications and try them out. It should not be hard to guess.

[patric-r]

It's a pity that you seem to refuse to answer my simple question:
Which application type corresponds to eclipse.exe of the last stable BIRT designer?

What does it make sense forcing me starting 20 application types when you know the answer or when it is somewhere configured? I started 3, but I don't see any difference.
And I made a guess: It must be default, "ReportDesigner". But I can't use it, because it's not working - have you tried creating a simple BIRT project?

We're planning contributing to this project, but we need your or the communities support and a perspective that this project is really alive again and to provide at least basic help/documentation of how to start developing. But compared to other open source projects, it seems to be much harder here...

[ruspl-afed]

Which application type corresponds to eclipse.exe of the last stable BIRT designer

Sorry, I do not understand this question :(

Are you trying to start BIRT Designer from sources?

What I can see is 2 product manifests with "designer" in its id

<product name="BIRT Report Designer All-in-one" uid="org.eclipse.birt.report.designer.all" application="org.eclipse.ui.ide.workbench" version="4.9.0.qualifier" useFeatures="true" includeLaunchers="true">

and

<product name="BIRT Report Designer RCP" uid="org.eclipse.birt.report.designer.rcp" application="org.eclipse.birt.report.designer.ui.rcp.DesignerApplication" version="4.9.0.qualifier" useFeatures="true" includeLaunchers="true">

So what we are talking about?

[patric-r]

I mean this one:
https://download.eclipse.org/birt/downloads/drops/I-R1-4.8.0-201805221921/birt-report-designer-all-in-one-4.8.0-20180522-win32.win32.x86_64.zip

Starting eclipse.exe shows me that what I call the "Report designer" (if you use other terms, it would be nice to learn them that we use the same terms).
In the "Report Design" perspective I have (in default configuration) in the bottom left the Navigator view, where I can create "Report design projects", see all files etc.

My question is: What do I have to do in order to start exactly this "application" from the BIRT development environment?

[ruspl-afed]

To start "All in One" you need to select

org.eclipse.birt.report.designer.ui.rcp.BIRT as a product
and
org.eclipse.ui.ide.workbench as an application

if something is not working as expected please open the dedicated issue to focus on concrete scenario

[wimjongman]

It's a pity that you seem to refuse to answer my simple question: Which application type corresponds to eclipse.exe of the last stable BIRT designer?

I am not going to spoon-feed you, my friend. We learn from frustration and bumping our noses. :) Good to see your frustration level rise ;)

What does it make sense forcing me starting 20 application types when you know the answer or when it is somewhere configured? I started 3, but I don't see any difference.

There are only a couple of them in the org.eclipse.birt namespace. Also, you have to clean your configuration before you start a new application otherwise you will not see any difference. Before you ask me how to do that, take a good look at the launch configuration form.

If an application does not run, you have to examine the messages. Most of the time there is a missing plugin or there are too many plugins.

I will organize a teams session for Monday. We can do a Q&A there.

Thank you for your testing efforts. Your tests were a driver for the recent developments. Try to stay positive.

[hvbtup]

The eclipse.exe from birt-report-designer-all-in-one is the right one.
If you build from source, there will be a zip in build\birt-packages\birt-report-all-in-one\target\products.
This is the right one (not the zip in build\birt-packages\birt-report-all-in-one\target).
You extract this somewhere and - given that you have a matching JRE (Java version and correct 32 or 64 bit) - you can start eclipse.exe in that directory.
Personally, I use a CMD script for this which ensures a correct environment, but this is not strictly necessary.
This eclipse.exe is the IDE.
Your first step should be to

1. Close the misleading welcome screen (which is suited for Java development),
2. Create a fresh workspace, and then open the Report Design perspective,
3. Create a new BIRT project "birt" (a directory where your rptdesign files go)
4. Create e.g. a folder for your libraries. I prefer a directory "res" inside "birt".
5. From the menu, choose Window / Preferences / Report Design / Resource append "/res" (corresponding to 4) so that it reads "/res".

Now you are ready to design reports.

7. Now your

[hvbtup]

BTW in my start script for the BIRT IDE, I set some environment variables and then call

start /D%~dp0birt %BIRT_EXE% -data %~dp0 -clean -vm "%JAVA_HOME%\bin\javaw.exe" -vmargs -Djava.util.logging.config.file=%~dp0logging.properties
(in a single line of course)

The logging.properties is only because I use java.util.logging from inside scripts in my reports.

[wimjongman]

I filed an issue to make life a little bit better for consumers: #694

[patric-r]

To start "All in One" you need to select

org.eclipse.birt.report.designer.ui.rcp.BIRT as a product and org.eclipse.ui.ide.workbench as an application

if something is not working as expected please open the dedicated issue to focus on concrete scenario

Thanks!
I was only able to choose between product and application configuration.
What I did: I chose org.eclipse.birt.report.designer.ui.rcp.BIRT project first, then switched the radio button to 'application' and chose org.eclipse.ui.ide.workbench - this finally started the right "application".
I need to do some more tests, but it looks like the original all-in-one designer package.

The viewer is not working, I know this had been discussed recently - is the github issue #649 the right one for this (because I don't get any stacktrace, just a 404 in the browser when doing a PDF preview)?

And BTW, I saw this in the error log: Project facet group group.birt.runtime has not been defined. It is used in plugin org.eclipse.birt.chart.integration.wtp.ui.

[patric-r]

@wimjongman
Thanks for your tutorial how to run the junit tests in the other issue!

I have question regarding the last step, making the plugin configuration for the tests identical to the birt-run launch configuration:
Is there a shortcut to export this from birt-run and import it in the test configuration (ideally, we would have a pre-packaged launch configuration for the tests as well).
What about the checkboxes below the plugin list?

[hvbtup]

I finally (think that) I understand what you mean.
Your question was not about how to start the BIRT IDE as a stand-alone program. Instead you meant how to start the BIRT IDE from inside the Eclipse IDE in order to debug e.g. changes to emitter plugins or whatever. Right?

[patric-r]

I finally (think that) I understand what you mean. Your question was not about how to start the BIRT IDE as a stand-alone program. Instead you meant how to start the BIRT IDE from inside the Eclipse IDE in order to debug e.g. changes to emitter plugins or whatever. Right?

Yes :) Sorry for the confusion.
Because that's what I need when doing changes/fixes - to test them locally.
The all-in-one-designer is perfect for this, because I can test everything (from design to 'emitter')
within one application.

How do you test/debug your local working copy changes?

[wimjongman]

Is there a shortcut to export this from birt-run and import it in the test configuration

I don't think there is a shortcut. The shortest way is maybe editing the file in a text editor. Launch configs are a moving target because of adding/removing future plugins. Launch configs can be different on different platforms. Therefore the developers should understand what makes them tick so that they can create/change them quickly.

[patric-r]

Especially because they are a moving target, it makes sense to commit them to the repo.
Otherwise, we need to rely on manual synchronization which will lead to increased effort (how do I know I need to adjust my plugins configuration?) and the developer needs to have a deeper understanding of the eclipse and BIRT ecosystem just to fix a trivial bug which is a hurdle for new contributors (like me for example).

[merks]

I fixed the reloading of the target platform already:

https://git.eclipse.org/c/oomph/org.eclipse.oomph.git/commit/?id=9ac25ae3ecb1145ee7c148436e19ba3e36849bfd

That's only in a nightly build so far...

But even with that, there is still rebuild upon startup and it's not clear what causes that. It happens even when to setup tasks are performed...

As I have suggested the problem isn't parent projects themselves but rather the fact that they show/include the entire tree of nested projects in them which can be avoided with filters, which I have started to work on...

[hvbtup]

I tried the eclipse installer and opened the workspace as Wim showed in the Q&A session. Eclipse shows 487 errors and ~100K warnings. I dont' understand why on my machine there are 487 errors, because I saw only 14 (?) on Wim's machine.
Maybe would make sense to share my screen in the next Q&A session. I certainly missed some tiny but important step.

BTW, Wim wanted to share the recorded Q&A session.

[wimjongman]

As I have suggested the problem isn't parent projects themselves but rather the fact that they show/include the entire tree of nested projects in them which can be avoided with filters, which I have started to work on...

Got it. I was confused about resource filters but I looked it up and now I'm good. I filed an issue for the resource filters:

Add resource filters to parent projects #699

[patric-r]

What is the recommend way / workflow to test own applications which use the BIRT runtimes againt the checked out working copy of BIRT without creating all BIRT jars by a full maven build and assigning those to the applications?

E.g. by adding "Required Projects" (birt projects) to the build path for such applications.
But which BIRT projects? How do you do it?

[ruspl-afed]

Assumptions:

1. you have a .target for your BIRT-based product (that includes BIRT features/bundels)
2. you have a workspace with you BIRT-based sources that uses this target
3. you have launch configuration that runs your BIRD-based product

Steps:

1. clone BIRT sources
2. Import BIRT projects that you would like to see in source form (perhaps, to fix something?)
3. normally, launch configuration will be auto switched from "target" to "workspace" instance of bundle but you can ensure that on "Plug-ins" tab of launch configuration.

[patric-r]

@ruspl-afed
Not quite. Our application is not an eclipse product. It's a plain java application. It doesn't have a GUI.
Currently, we add the following BIRT/eclipse jars to the classpath:

Afterwards, we start the platform, load the design handle and create the runAndRender task:

    Platform.startup(engineConfig_);
    this.reportEngine_ = new ReportEngine(engineConfig_);
    reportEngine_.changeLogLevel(Level.WARNING);
    this.pdfRenderOption_.setOutputFormat(FILE_EXTENSION);
    this.reportRunnable_ = reportEngine_.openReportDesign(this.reportDesign_);
    this.runAndRenderTask_ = reportEngine_.createRunAndRenderTask(reportRunnable_);		 

For each to-be-created PDF we do:

    this.runAndRenderTask_.setRenderOption(pdfRenderOption_);
    runAndRenderTask_.setLocale(Locale.GERMAN);
    this.runAndRenderTask_.run();

[hvbtup]

In our application we work similar to @patric-r. We are using the "birt-runtime" here (without OSGI). But instead of explicitly adding single JARs we just add ReportEngine/lib/* to the classpath. This includes all the files in that directory.

This requires the Maven build of course.

But to just test modifications of the BIRT source, we do not need to do it this way:
Our reports are written in such a way that they can be tested from within the BIRT designer (the DB connection info is supplied using environment variables then). So I can test from within Eclipse instead of from our application.
Soemtimes, when trying to fix a bug, I try to create a simple test case using only the builtin sample database. This makes testing the effect of source code changes even easier and allows other BIRT developers to see the effect as well.

[patric-r]

@hvbtup
Glad to hear that we're not the only one integrating BIRT that way.
Thanks for the info for the non-OSGI approach / BIRT runtime, I will definitely look into that.

You're completely right, we do it in a similar way that we can test everything in the BIRT designer (except that we don't use DB connections within BIRT, we decoupled the db model from the BIRT design model, so our process is roughly -> Export relevant data from DB to XML, start our application which invokes BIRT in the above way and the design uses the xml as a datasource).

However, I know that there are performance issues and memory leaks in the BIRT engine (at least in the version we're using) which only appear when creating many PDFs within one ReportEngine.
In order to fix them it is much more easier invoking the rendering task multiple times automatically instead of manually clicking 20 times on the preview button (and the viewer is currently not working).

Is there a way to achieve this?
I would love to improve memory consumption and maybe even rendering performance.

[merks]

Note that JDT's increment compiler automatically creates the <project>/target/classes/ folder for each project so everywhere that you need a jar you could instead point at this corresponding folder...

[merks]

Here's another trick to build a class path. Go to any of the JUnit tests and do Run as (or DebugAs) -> JUnit test. Then open the Debug perspective (upper right button with +), and you see the terminated process. Right click (context menu) the nested process object and use Properties... Here you can see the command line used to launch the test. This has the full class path of everything used by that test.

[wimjongman]

We have created a video that explains the full cycle of contributing to BIRT.

https://youtu.be/FqfrG2I0AIw

00:00 Intro
00:40 Fork the main BIRT repository
02:00 Download the Eclipse Installer
03:10 Start Installation
05:20 Start Eclipse
05:50 Automatically configure the development environment
09:00 Running BIRT with your changes
12:00 Running Unit Tests
14:00 Making and pushing a change
16:00 Creating a Pull Request

[hvbtup]

However, I know that there are performance issues and memory leaks in the BIRT engine (at least in the version we're using) which only appear when creating many PDFs within one ReportEngine. In order to fix them it is much more easier invoking the rendering task multiple times automatically instead of manually clicking 20 times on the preview button (and the viewer is currently not working).

Is there a way to achieve this? I would love to improve memory consumption and maybe even rendering performance.

I don't know about BIRT specific memory leaks and performance issues.
But in the process model that we are running BIRT reports (sevearl Java processes, each one processes several reports single-threaded) I noticed that

• BIRT needs quite a lot of memory when generating a complex PDF report. This is ok though.
• So our JVM processes are created with eg -xmx 512m or 1024m (for larger reports) (don't know the exact syntax of the memory parameters right now)
• The JVM seemts to perform a "big" GC only when the heap reaches its maximum (by default).
• The GC then takes a lot of time
• Having too low -xmx values causes the JVM to perform GC too often (e.g. moving a lot of small objects only to get tiny new bits of free space). This causes hight CPU load and slow reports.
• Having too high -xmx values can cause out-of-memory errors on the OS side (or in Java) with this multi-process model because by default the JVM does not give back unused memory to the OS.
• Before we switched to the multi-process report execution model, we used a mult-threaded single process. The reason why we switched was because some reports (e.g. containing many charts or images) need a lot of memory, while others only need a little bit. The little report may run into OutOfMemoryError when allocating a few dozen bytes because the big report allocated hundres of MB. Users don't understand this. And once you have OOM, the only thing you can safely do with your JVM is to exit the process. So one report could cause failures in other reports. This risk is minimized with the multi-process execution model.

In order to do load testing or performance testing, you should definitely use test scripts or programs with loops which start e.g. 1000 reports and wait until all 1000 are finished.

[colinsutton]

We run all reports with -d64 -Xmx6200m -Xms512m (though only one huge report needs the 6GB), using birt (4.5 and 4.8) under wildfly on dedicated windows servers with multiple users.

[hvbtup]

tried the eclipse installer and opened the workspace as Wim showed in the Q&A session. Eclipse shows 487 errors and ~100K warnings.

I was able to solve this. Now I only have one remaining error message:

Host bundle 'org.eclipse.birt.data.oda.pojo.ui' cannot be resolved

in org.eclipse.birt.data.oda.pojo.ui/META-INF/MANIFEST.mf

This seems to be a basic beginner's eclipse issue. I think it is worth mentioning.

What I had to do:

1. Choose the JRE that came with the eclipse installer in Preferences / Java / Installed JREs

I have to repeat this every time I start eclipse.exe, as it seems. Otherwise Eclipse finds and uses the JRE 11 in c:\prog\jdk... and that causes a lot of "Failed to init ct.sym" errors.

For birt-runtime-test (inside BIRT Tests) and org.eclipse.birt.axis.overlay (inside BIRT Bundles) I had to explicitly choose JavaSE-11 in the build path.

[wimjongman]

"Failed to init ct.sym" is a strange issue. I don't know what causes it.

[ruspl-afed]

@hvbtup Please try to configure installer to use the desired JRE instead of fighting with it

[hvbtup]

I think that's what I did. But I had Product Version 2021-12 selected and my Java 11+ VM is a bit older than yours.
I'll try again, this time with 2021-09:

[hvbtup]

https://stackoverflow.com/questions/63446925/the-project-was-not-built-due-to-failed-to-init-ct-sym-for-c-program-files-jav helped me to solve this. But I still don't understand the error.