Skip to content
This repository has been archived by the owner on Sep 23, 2024. It is now read-only.

Tap stuck at select query #212

Open
piyushdatazip opened this issue Mar 29, 2023 · 3 comments
Open

Tap stuck at select query #212

piyushdatazip opened this issue Mar 29, 2023 · 3 comments
Labels
question Further information is requested

Comments

@piyushdatazip
Copy link

Describe the bug
I was running tap following the config and catalog guidelines, was able to pull the streams but sync was stuck

To Reproduce
Steps to reproduce the behavior:

tap-postgres -c config.json --catalog catalog.json -p properties.json

Expected behavior
It should work!

Logs

2023-03-29 10:01:44 [DEBUG] - time=2023-03-29 10:01:44 name=tap_postgres level=INFO message=Selected streams: ['public-contact_data_source'] 

2023-03-29 10:01:44 [DEBUG] - time=2023-03-29 10:01:44 name=tap_postgres level=INFO message=No streams marked as currently_syncing in state file
time=2023-03-29 10:01:44 name=tap_postgres level=INFO message=Beginning sync of stream(public-contact_data_source) with sync method(full)

2023-03-29 10:01:44 [DEBUG] - time=2023-03-29 10:01:44 name=tap_postgres level=INFO message=Stream public-contact_data_source is using full_table replication

2023-03-29 10:01:44 [WARN ] - Unknown Singer output line type: ACTIVATE_VERSION [map[stream:public-contact_data_source type:ACTIVATE_VERSION version:1.680084104966e+12]]
2023-03-29 10:01:45 [DEBUG] - time=2023-03-29 10:01:45 name=tap_postgres level=INFO message=Current Server Encoding: UTF8

2023-03-29 10:01:45 [DEBUG] - time=2023-03-29 10:01:45 name=tap_postgres level=INFO message=Current Client Encoding: UTF8

2023-03-29 10:01:45 [DEBUG] - time=2023-03-29 10:01:45 name=tap_postgres level=INFO message=hstore is UNavailable
time=2023-03-29 10:01:45 name=tap_postgres level=INFO message=Beginning new Full Table replication 1680084104966
time=2023-03-29 10:01:45 name=tap_postgres level=INFO message=select SELECT  "contact_data_source_id" , "contact_id" , "contact_list_id" ,CASE WHEN  "date_created"  < '0001-01-01 00:00:00.000' OR  "date_created"  > '9999-12-31 23:59:59.999' THEN '9999-12-31 23:59:59.999' ELSE  "date_created"  END AS  "date_created" , "id" , "is_deleted" , "new_contact_id" , "new_lead_id" , "org_id" , "other_properties" , xmin::text::bigint
                                      FROM "public"."contact_data_source"
                                     ORDER BY xmin::text ASC with itersize 20000

Your environment

  • Version of tap: 2.0.0
  • Version of python: 3.10
@piyushdatazip piyushdatazip added the bug Something isn't working label Mar 29, 2023
@Samira-El
Copy link
Contributor

Hi, it's not stuck, depending on the size of your table, the query might run for a long time.

@Samira-El Samira-El added question Further information is requested and removed bug Something isn't working labels Mar 30, 2023
@piyushdatazip
Copy link
Author

I monitored it for like 15 mins, it still didn't move forward no logs, no records!

@Samira-El
Copy link
Contributor

There will be no logs from the tap until this query finishes, but it should be processing rows and you would be getting streams in the stdout.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants