Unable to to read data for source google sheets

[DeepakDonde-GWL]

The task is to sync data from google sheets to Postgres database. But facing read issues with google sheets.

Code :

import airbyte as ab
import json

source = ab.Source=ab.get_source("source-google-sheets")

account_string=json.dumps({
  "type": "service_account",
  "project_id": "test-443710",
  "private_key_id": "20f0c1c481eb4b28aec6adc8b432960d14c54",
  "private_key": "-----BEGIN PRIVATE KEY-----\nMIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDtUTkajQwEl90n\nUb9zTZ+pxsHfBG2CBpkJgm4A4U3NhP6aqu8EtnVskqfzFKGuBdrshVMdYbR/\nwpNZCDGEMjx56/4KQGnSM0BdlKaAANod3rblhe/HYv9eL5VB+NAwVnjMkc2O9CaB\nBAMWlm+RDwKBgQCohWMZc9+nS8GHt7g/5EEjme9IG4ZSrPTPX/fmb91qGl8McEqnC6+G\nyOyVuH/cXqgnKJolbzanBroJTLmyEr66Fo9PLiiMwwKBgBbQO8lwSfZ/7Vm2ildE\nGo8vewzPBUFE6v75Y3KdKz+PLONiaf4fGx8YFcHDoJEfgATEoU/Ipf8xjd+Jmukd\nSGDvAodHLtSnoinCkUdA2McM8TBrxDrl/3Ir2oSvchNTkXOIx4osxqFo+C48MxMa\nkB0tyJvBHLMoboX/TUxG5efo\n-----END PRIVATE KEY-----\n",
  "client_email": "[email protected]",
  "client_id": "111345679374858679179",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/airbyte%40airbyte-443710.iam.gserviceaccount.com",
  "universe_domain": "googleapis.com"
}
)

source.set_config(config={
     "spreadsheet_id": "1MFrk0uOnoVrMvw7-QrfrwDk",
     "credentials": {"auth_type":"Service","service_account_info":account_string}
})

source.check()

source.select_all_streams()
readResult:ab.ReadResult = source.read()
print("readresult",readResult)
userdf= readResult["users"].to_pandas()
print(userdf)

source.check() returns success.

But getting below error when try to read stream:

ERROR: Something went wrong in the connector. See the logs for more details.
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
                                      Sync Progress: source-google-sheets -> DuckDBCache                                       

───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Traceback (most recent call last):
  File "/home/gwl/deepak/Learnings/PyAirbyte/first/main.py", line 46, in <module>
    readResult:ab.ReadResult = source.read()
  File "/home/gwl/.local/lib/python3.10/site-packages/airbyte/sources/base.py", line 663, in read
    result = self._read_to_cache(
  File "/home/gwl/.local/lib/python3.10/site-packages/airbyte/sources/base.py", line 747, in _read_to_cache
    cache._write_airbyte_message_stream(  # noqa: SLF001  # Non-public API
  File "/home/gwl/.local/lib/python3.10/site-packages/airbyte/caches/base.py", line 306, in _write_airbyte_message_stream
    cache_processor.process_airbyte_messages(
  File "/home/gwl/.local/lib/python3.10/site-packages/airbyte/shared/sql_processor.py", line 268, in process_airbyte_messages
    for message in messages:
  File "/home/gwl/.local/lib/python3.10/site-packages/airbyte/sources/base.py", line 576, in _read_with_catalog
    yield from progress_tracker.tally_records_read(message_generator)
  File "/home/gwl/.local/lib/python3.10/site-packages/airbyte/progress.py", line 261, in tally_records_read
    for count, message in enumerate(messages, start=1):
  File "/home/gwl/.local/lib/python3.10/site-packages/airbyte/_connector_base.py", line 453, in _execute
    raise exc.AirbyteConnectorFailedError(
airbyte.exceptions.AirbyteConnectorFailedError: Connector failed. (AirbyteConnectorFailedError)
------------------------------------------------------------
AirbyteConnectorFailedError: Connector failed.
    Please review the log file for more information.
    Connector Name: 'source-google-sheets'
    Exit Code: 1
    Log file: /tmp/airbyte/logs/source-google-sheets/source-google-sheets-log-JE8PJAJSY.log

.log file:

2024-12-04 16:55:03 - INFO - Check succeeded
2024-12-04 16:55:05 - INFO - Running discovery on sheet 1MFrk0uOV6UFJ1F9kO0no-LXT3fKnfrwDk
2024-12-04 16:55:09 - INFO - Finished syncing spreadsheet 1MFrk0uOV6UFJ1F9kO0w7-QrfrwDk
2024-12-04 16:55:09 - INFO - ConnectorStateManager.__init__() got an unexpected keyword argument 'stream_instance_map'
Traceback (most recent call last):
  File "/home/gwl/deepak/Learnings/PyAirbyte/first/.venv-source-google-sheets/bin/source-google-sheets", line 8, in <module>
    sys.exit(run())
  File "/home/gwl/deepak/Learnings/PyAirbyte/first/.venv-source-google-sheets/lib/python3.10/site-packages/source_google_sheets/run.py", line 15, in run
    launch(source, sys.argv[1:])
  File "/home/gwl/deepak/Learnings/PyAirbyte/first/.venv-source-google-sheets/lib/python3.10/site-packages/airbyte_cdk/entrypoint.py", line 234, in launch
    for message in source_entrypoint.run(parsed_args):
  File "/home/gwl/deepak/Learnings/PyAirbyte/first/.venv-source-google-sheets/lib/python3.10/site-packages/airbyte_cdk/entrypoint.py", line 122, in run
    yield from map(AirbyteEntrypoint.airbyte_message_to_string, self.read(source_spec, config, config_catalog, state))
  File "/home/gwl/deepak/Learnings/PyAirbyte/first/.venv-source-google-sheets/lib/python3.10/site-packages/airbyte_cdk/entrypoint.py", line 164, in read
    for message in self.source.read(self.logger, config, catalog, state):
  File "/home/gwl/deepak/Learnings/PyAirbyte/first/.venv-source-google-sheets/lib/python3.10/site-packages/source_google_sheets/source.py", line 244, in read
    yield from self._read(logger, config, catalog, state)
  File "/home/gwl/deepak/Learnings/PyAirbyte/first/.venv-source-google-sheets/lib/python3.10/site-packages/source_google_sheets/source.py", line 166, in _read
    state_manager = ConnectorStateManager(stream_instance_map=stream_instances, state=state or {})
TypeError: ConnectorStateManager.__init__() got an unexpected keyword argument 'stream_instance_map'
2024-12-04 16:55:09 - ERROR - Something went wrong in the connector. See the logs for more details.
2024-12-04 16:55:09 - INFO - {"type":"TRACE","trace":{"type":"ERROR","emitted_at":1733311509882.466,"error":{"message":"Something went wrong in the connector. See the logs for more details.","internal_message":"ConnectorStateManager.__init__() got an unexpected keyword argument 'stream_instance_map'","stack_trace":"Traceback (most recent call last):\n  File \"/home/gwl/deepak/Learnings/PyAirbyte/first/.venv-source-google-sheets/bin/source-google-sheets\", line 8, in <module>\n    sys.exit(run())\n  File \"/home/gwl/deepak/Learnings/PyAirbyte/first/.venv-source-google-sheets/lib/python3.10/site-packages/source_google_sheets/run.py\", line 15, in run\n    launch(source, sys.argv[1:])\n  File \"/home/gwl/deepak/Learnings/PyAirbyte/first/.venv-source-google-sheets/lib/python3.10/site-packages/airbyte_cdk/entrypoint.py\", line 234, in launch\n    for message in source_entrypoint.run(parsed_args):\n  File \"/home/gwl/deepak/Learnings/PyAirbyte/first/.venv-source-google-sheets/lib/python3.10/site-packages/airbyte_cdk/entrypoint.py\", line 122, in run\n    yield from map(AirbyteEntrypoint.airbyte_message_to_string, self.read(source_spec, config, config_catalog, state))\n  File \"/home/gwl/deepak/Learnings/PyAirbyte/first/.venv-source-google-sheets/lib/python3.10/site-packages/airbyte_cdk/entrypoint.py\", line 164, in read\n    for message in self.source.read(self.logger, config, catalog, state):\n  File \"/home/gwl/deepak/Learnings/PyAirbyte/first/.venv-source-google-sheets/lib/python3.10/site-packages/source_google_sheets/source.py\", line 244, in read\n    yield from self._read(logger, config, catalog, state)\n  File \"/home/gwl/deepak/Learnings/PyAirbyte/first/.venv-source-google-sheets/lib/python3.10/site-packages/source_google_sheets/source.py\", line 166, in _read\n    state_manager = ConnectorStateManager(stream_instance_map=stream_instances, state=state or {})\nTypeError: ConnectorStateManager.__init__() got an unexpected keyword argument 'stream_instance_map'\n","failure_type":"system_error","stream_descriptor":null}}}