Ga4 support

[LindaLawton]

Update for Ga4 alpha support is on going and can be found in the following branch

Add-ga4-event-support

[JanZeman]

Hi @LindaLawton, I can see this repo did not get any update for quite some time. Do you plan to further develop on it? Or do you know any alternatives? I am curious how to most efficiently implement GA4 reporting from within .NET. I tested your GA4 branch and after very minor modifications it worked for me but I always get only the events whereas I am also very interested in users / demographics / etc. So far my users are always 0. This seems to by 'by design' of Measurement Protocol but so far I did not find out how to mimic the behavior implemented i.e. in Firebase Android SDK. Intuitively I start searching on auto-reported events user_engagement or first_visit but before burning a lot of research time I thought I also ask here. Any hint would be very appreciated. Best regards, Jan

[LindaLawton]

This library will only work with UA analytics not with GA4.

As the new version of Google analytics GA4 uses a different form for measurement protocol my plan was to create new project for that.

I was hoping to wait until GA4 is out of beta to start creating the new project.

As far as i know i am the only one working on measurement protocol libraries for .Net

[JanZeman]

Hi @LindaLawton, thank your for the reply. Yes, I am aware it is based on new MP. I understood it that way from the beginning that the given branch was there only for the alpha experiment. I decided to stay with the older protocol for few more months but will observe. If you start a new project I would consider some kind of contribution to that repo, from testing, potential bug-fixing over to some missing features implementation. Have a nice day, Jan

[LindaLawton]

I like the idea of having it working similar to Firebase Android SDK. It might not be a bad idea to start the project now even if the protocol is still in beta.

[LindaLawton]

@JanZeman if your still interested. I may have a beta version ready shortly. I need to add all the standard events, the first version requires that you build the events up yourself.

[JanZeman]

Hi @LindaLawton, that is a great news, thank you for following up! I am surely still interested, even more than before as I recently noticed this message from Google: "Universal Analytics will no longer process new data in standard properties beginning July 1, 2023. Prepare now by setting up and switching over to a Google Analytics 4 property." I have couple of very busy days in front of me but next week I put some time aside. Please instruct me with details. Will you push your changes to 'Add-ga4-event-support' branch? Or any other? I am not GA expert but will try my best. From my little experiment in February I remember I was not even able to see active users on my testing GA4 property. That would be my first, I believe natural, goal. If I remember correctly I was able to see one or two custom events there. When you talk about 'standard events', is it a synonym of '[GA4] Automatically collected events' listed here?

[LindaLawton]

Please instruct me with details. Will you push your changes to 'Add-ga4-event-support' branch? Or any other?

I am open to opinions on that. Its a completely different system so i am leaning towards a completely new repo. I can point to that from this one.

When you talk about 'standard events', is it a synonym of '[GA4] Automatically collected events' listed here?

Yes all those.

background

I have two options or i want to have two options. One you configure it at start up and then i have a custom middleware that listens for all the calls coming in. If the controller is tagged with a [GoogleAnalytics] attribute then it will try to build the event on its own. Event name would be that of the controller, properties would be properties sent along to the method.

The second option is build it yourself. A quick example would be

     var postData = new Event("TestEVent");
    postData.AddParameter("TestString", "test");
    postData.AddParameter("testInt", 1);
    postData.AddItem("test", 1);
    postData.AddItem("test2", 1);
    postData.AddItem("test3", 1);

    var data = new EventRequest()
    {
        ClientId = "test",
        Events = new []
        {
            postData
        }
    };

    _service.Send(data);

Note service is an IMeasurementService service that you can grab though DI.

Option two appears to work now. Option one i am still testing.

This is what I am working on now standard events built for these so that at least the name is set for you and the properties that you are allowed to use.

so that you could just call _service.SendAdClick(eventId);

prebuilt events

What I was thinking of doing is building them all as follows So EventBuilders just builds them for you with the required parameters.

  var data = new EventPayload()
    {
        ClientId = "test",
        Events = new Event[]
        {
            EventBuilders.BuildAdClickEvent("adClickIdTest")
        }
    };

    _service.Send(data);

We may not be allowed to tough. I am have a question off to the GA team

{
  "validationMessages": [ {
    "fieldPath": "events",
    "description": "Event at index: [0] has name [ad_click] which is reserved.",
    "validationCode": "NAME_RESERVED"
  } ]
}

Once i clean it up I can push up an alpha version.

[JanZeman]

That all sounds very reasonable. And yes, completely new repo would make probably more sense. Please let me know about the alpha version as I said I will put enough time aside to test it.

[LindaLawton]

Im cleaning it up a bit. Im going to just have the custom builder to start with I can add other ones while we do some testing to the initial alpha release.

The main thing i need help with testing is that it appears like it does with android and ios. I dont have android or ios super powers so i can test them.

[JanZeman]

I do have Xamarin based solution, as of now with Windows and Android implementation. iOS waits there to be picked up later. So I can surely test on Android.

[LindaLawton]

Here you go. Let me know if you need me to build you a nuget package for it GoogleAnalytics.Ga4.Dotnet.Sdk

[JanZeman]

Hi Linda,

I wrote the below text as a comment to PR I planned to create on your new repository. Will you please give me the permissions there? If you do, I will push my branch to GoogleAnalytics.Ga4.Dotnet.Sdk and create PR there.

I finally found couple of hours. At this step my goal was to see at least one event that originates from the SDK code on my "Learn GA4" Analytics property. I set up one Android and one Web stream there but today used only the Web one as I saw your appsettings.json expects exactly this one.

I removed the debug param in WeatherForecastController to use "/mp/collect" instead of "/debug/mp/collect" and hoped some event appears on my GA4. That however was not the case. I had to set up few tools to be able to observer bit more closer what is happening. Namely Charles proxy and its SSL certificate and Postman where I tried to mimic the requests.

Thank to those tools I discovered 2 problems:

1. timestamp_micros value was set in seconds but must be in microseconds.
2. There was a tiny typo in URL formatting that added the $ character which made the request invalid.

The PR fixes the above plus some other smaller typos and formatting.

I am attaching few screenshots, mainly the one from GA. The "test_event" originates from SDK.

The other images are from Charles proxy capturing the traffic between SDK and Analytics server.

I also touched unit tests that did not compile at the moment. I got them to green but did not make it part of this PR as I presume your plan is to extent mocks usage whereas my quick fix was 'gaOptions', i.e. these became rather integration tests communicating with "/debug/mp/collect".

My big goal would be to see the Users being captured in the Audience. This is still empty but we know that this is not in the scope of those first commits.

Undskyld, hvis det er for personligt, men er du måske fra Danmark? ;)

Best regards
Jan

[LindaLawton]

You should be able to crate a fork of the project and then push it back as a pull request.

I had an issue with sending the data as well if was a new google analytics account and it was a few days before the hits started showing up. When did you create the account?

Ja! jeg er også fra denmark :)

[JanZeman]

Oh, great. I am sorry, not contributed to that many GitHub projects. Yes, with forking it works, this is the PR: DAIMTO/GoogleAnalytics.Ga4.Dotnet.Sdk#1. I copy & pasted the relevant info there.

The account I tested with is few years old but the 'Test GA4' property only from January 2022. It might be that it played some role.

Sjovt :) Jeg er faktisk Tjekke, men har boet i Danmark i nogle år. Mit talte dansk er helt sikkert ikke perfekt ;)

I will try to follow up on new repo. I will have to study bit more about the Audience. As I mentioned I would like to see every visitor recorded there usage. I also do not understand why GA server returns 204 and not 200, plus I have 100+1 more of other questions :)

[LindaLawton]

It always responds with 204 because there is no content returned. I need to have a chat with the GA team about that one seems weird.

Jeg er faktisk amerikansk, Jeg flytte til Denmark i 1998. Jeg er heller ikke godt til at skriver dansk.

[JanZeman]

I agree, the 204 is bit confusing. Accepted reply with the attached video in this SO issue suggests using payload instead of body. That is however not required based on my experience.