feat(observability): more descriptive and value adding spans

[odeke-em]

This change adds more descriptive and value adding spans to replace the generic CloudSpanner.ReadWriteTransaction. With this change, we add new spans:

• CloudSpanner.Database.run_in_transaction
• CloudSpanner.execute_pdml
• CloudSpanner.execute_sql
• CloudSpanner.execute_update

Exhibit

Given this code

def tx_dql_select(tx):
    try:
        tx.execute_update("DELETE FROM Singers WHERE 1=1")
    except:
        pass

    tx.execute_update(
        'INSERT INTO Singers(SingerId, FirstName, LastName) VALUES(1000, "Bryan", "Adams"), (2000, "Slash", "GunsNRoses")'
    )
    tx.update(
        "Singers",
        columns=["SingerId", "FirstName"],
        values=[["1", "Bryan"], ["2", "Slash"]],
    )
    rows = tx.execute_sql("SELECT * FROM Singers")
    for row in rows:
        print(row)

we now have this exhibit

Without gRPC instrumentation

With gRPC instrumentation

[odeke-em]

Kindly help me take a look @harshachinta and help me run the bots afresh!

[odeke-em]

@harshachinta please take a look.

[odeke-em]

@harshachinta kindly help me run the bots one more time, I sent up a fix!

[harshachinta]

I think we can do a much simpler thing instead of adding db_name manually at every place.
We have configured a @Property for observability_options in database.py

python-spanner/google/cloud/spanner_v1/database.py

Line 1111 in 054a186

def observability_options(self):

Can we add a new field, db_name, to observability_options if it already exists? If the customer does not provide any observability_options, we can create a new dictionary with this field. With this we need not do it at multiple places, WDYT?

[harshachinta]

The problem i see here is with every retry there will be a new span parent CloudSpanner.execute_partitioned_pdml which does not indicate that a retry is happening.
Instead we should capture the retries with in only one parent span CloudSpanner.execute_partitioned_pdml.