Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Start using apify-shared for general consts and utils #132

Merged
merged 8 commits into from
Jul 28, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,12 @@ filename =
./scripts/*.py,
./src/*.py,
./tests/*.py
# Todo: remove "src/apify_client/consts.py: F401" once consts are removed from the package
per-file-ignores =
docs/*: D
scripts/*: D
tests/*: D
src/apify_client/consts.py: F401

# Google docstring convention + D204 & D401
docstring-convention = all
Expand Down
2 changes: 1 addition & 1 deletion .isort.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ line_length = 150
use_parentheses = True
multi_line_output = 3
sections = FUTURE,STDLIB,THIRDPARTY,FIRSTPARTY,LOCALFOLDER
known_first_party = apify_client
known_first_party = apify_client, apify_shared
255 changes: 0 additions & 255 deletions docs/docs.md
Original file line number Diff line number Diff line change
Expand Up @@ -6155,258 +6155,3 @@ List all webhook dispatches of a user.
* **Return type**

[`ListPage`](#listpage)

***

### [](#listpage) ListPage

A single page of items returned from a list() method.

#### Instance attributes

Name | Type | Description
---- | ---- | -----------
`items` | `list` | List of returned objects on this page
`offset` | `int` | The limit on the number of returned objects offset specified in the API call
`limit` | `int` | The offset of the first object specified in the API call
`count` | `int` | Count of the returned objects on this page
`total` | `int` | Total number of objects matching the API call criteria
`desc` | `bool` | Whether the listing is descending or not

***

### [](#actorjobstatus) ActorJobStatus

Available statuses for actor jobs (runs or builds).

* [READY](#actorjobstatus-ready)
* [RUNNING](#actorjobstatus-running)
* [SUCCEEDED](#actorjobstatus-succeeded)
* [FAILED](#actorjobstatus-failed)
* [TIMING\_OUT](#actorjobstatus-timing\_out)
* [TIMED\_OUT](#actorjobstatus-timed\_out)
* [ABORTING](#actorjobstatus-aborting)
* [ABORTED](#actorjobstatus-aborted)

***

#### [](#actorjobstatus-ready) `ActorJobStatus.READY`

Actor job initialized but not started yet

***

#### [](#actorjobstatus-running) `ActorJobStatus.RUNNING`

Actor job in progress

***

#### [](#actorjobstatus-succeeded) `ActorJobStatus.SUCCEEDED`

Actor job finished successfully

***

#### [](#actorjobstatus-failed) `ActorJobStatus.FAILED`

Actor job or build failed

***

#### [](#actorjobstatus-timing_out) `ActorJobStatus.TIMING_OUT`

Actor job currently timing out

***

#### [](#actorjobstatus-timed_out) `ActorJobStatus.TIMED_OUT`

Actor job timed out

***

#### [](#actorjobstatus-aborting) `ActorJobStatus.ABORTING`

Actor job currently being aborted by user

***

#### [](#actorjobstatus-aborted) `ActorJobStatus.ABORTED`

Actor job aborted by user

***

### [](#actorsourcetype) ActorSourceType

Available source types for actors.

* [SOURCE\_FILES](#actorsourcetype-source\_files)
* [GIT\_REPO](#actorsourcetype-git\_repo)
* [TARBALL](#actorsourcetype-tarball)
* [GITHUB\_GIST](#actorsourcetype-github\_gist)

***

#### [](#actorsourcetype-source_files) `ActorSourceType.SOURCE_FILES`

Actor source code is comprised of multiple files

***

#### [](#actorsourcetype-git_repo) `ActorSourceType.GIT_REPO`

Actor source code is cloned from a Git repository

***

#### [](#actorsourcetype-tarball) `ActorSourceType.TARBALL`

Actor source code is downloaded using a tarball or Zip file

***

#### [](#actorsourcetype-github_gist) `ActorSourceType.GITHUB_GIST`

Actor source code is taken from a GitHub Gist

***

### [](#webhookeventtype) WebhookEventType

Events that can trigger a webhook.

* [ACTOR\_RUN\_CREATED](#webhookeventtype-actor\_run\_created)
* [ACTOR\_RUN\_SUCCEEDED](#webhookeventtype-actor\_run\_succeeded)
* [ACTOR\_RUN\_FAILED](#webhookeventtype-actor\_run\_failed)
* [ACTOR\_RUN\_TIMED\_OUT](#webhookeventtype-actor\_run\_timed\_out)
* [ACTOR\_RUN\_ABORTED](#webhookeventtype-actor\_run\_aborted)
* [ACTOR\_RUN\_RESURRECTED](#webhookeventtype-actor\_run\_resurrected)
* [ACTOR\_BUILD\_CREATED](#webhookeventtype-actor\_build\_created)
* [ACTOR\_BUILD\_SUCCEEDED](#webhookeventtype-actor\_build\_succeeded)
* [ACTOR\_BUILD\_FAILED](#webhookeventtype-actor\_build\_failed)
* [ACTOR\_BUILD\_TIMED\_OUT](#webhookeventtype-actor\_build\_timed\_out)
* [ACTOR\_BUILD\_ABORTED](#webhookeventtype-actor\_build\_aborted)

***

#### [](#webhookeventtype-actor_run_created) `WebhookEventType.ACTOR_RUN_CREATED`

The actor run was created

***

#### [](#webhookeventtype-actor_run_succeeded) `WebhookEventType.ACTOR_RUN_SUCCEEDED`

The actor run has succeeded

***

#### [](#webhookeventtype-actor_run_failed) `WebhookEventType.ACTOR_RUN_FAILED`

The actor run has failed

***

#### [](#webhookeventtype-actor_run_timed_out) `WebhookEventType.ACTOR_RUN_TIMED_OUT`

The actor run has timed out

***

#### [](#webhookeventtype-actor_run_aborted) `WebhookEventType.ACTOR_RUN_ABORTED`

The actor run was aborted

***

#### [](#webhookeventtype-actor_run_resurrected) `WebhookEventType.ACTOR_RUN_RESURRECTED`

The actor run was resurrected

***

#### [](#webhookeventtype-actor_build_created) `WebhookEventType.ACTOR_BUILD_CREATED`

The actor build was created

***

#### [](#webhookeventtype-actor_build_succeeded) `WebhookEventType.ACTOR_BUILD_SUCCEEDED`

The actor build has succeeded

***

#### [](#webhookeventtype-actor_build_failed) `WebhookEventType.ACTOR_BUILD_FAILED`

The actor build has failed

***

#### [](#webhookeventtype-actor_build_timed_out) `WebhookEventType.ACTOR_BUILD_TIMED_OUT`

The actor build has timed out

***

#### [](#webhookeventtype-actor_build_aborted) `WebhookEventType.ACTOR_BUILD_ABORTED`

The actor build was aborted

***

### [](#metaorigin) MetaOrigin

Possible origins for actor runs, i.e. how were the jobs started.

* [DEVELOPMENT](#metaorigin-development)
* [WEB](#metaorigin-web)
* [API](#metaorigin-api)
* [SCHEDULER](#metaorigin-scheduler)
* [TEST](#metaorigin-test)
* [WEBHOOK](#metaorigin-webhook)
* [ACTOR](#metaorigin-actor)

***

#### [](#metaorigin-development) `MetaOrigin.DEVELOPMENT`

Job started from Developer console in Source section of actor

***

#### [](#metaorigin-web) `MetaOrigin.WEB`

Job started from other place on the website (either console or task detail page)

***

#### [](#metaorigin-api) `MetaOrigin.API`

Job started through API

***

#### [](#metaorigin-scheduler) `MetaOrigin.SCHEDULER`

Job started through Scheduler

***

#### [](#metaorigin-test) `MetaOrigin.TEST`

Job started through test actor page

***

#### [](#metaorigin-webhook) `MetaOrigin.WEBHOOK`

Job started by the webhook

***

#### [](#metaorigin-actor) `MetaOrigin.ACTOR`

Job started by another actor run
4 changes: 0 additions & 4 deletions docs/res/sphinx-config/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,3 @@
:members:
.. automodule:: apify_client.clients.resource_clients
:members:
.. autoclass:: apify_client._utils.ListPage
:members:
.. automodule:: apify_client.consts
:members:
vdusek marked this conversation as resolved.
Show resolved Hide resolved
5 changes: 3 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description = "Apify API client for Python"
readme = "README.md"
license = {text = "Apache Software License"}
authors = [
{name = "Apify Technologies s.r.o.", email = "[email protected]" }
{name = "Apify Technologies s.r.o.", email = "[email protected]"},
]
keywords = ["apify", "api", "client", "scraping", "automation"]

Expand All @@ -23,7 +23,8 @@ classifiers = [

requires-python = ">=3.8"
dependencies = [
"httpx >= 0.24.1"
"apify-shared ~= 1.0.0",
"httpx >= 0.24.1",
]

[project.optional-dependencies]
Expand Down
2 changes: 1 addition & 1 deletion src/apify_client/_errors.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

import httpx

from ._utils import ignore_docs
from apify_shared.utils import ignore_docs


class ApifyClientError(Exception):
Expand Down
17 changes: 6 additions & 11 deletions src/apify_client/_http_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,17 +9,12 @@

import httpx

from apify_shared.types import JSONSerializable
from apify_shared.utils import ignore_docs, is_content_type_json, is_content_type_text, is_content_type_xml

from ._errors import ApifyApiError, InvalidResponseBodyError, _is_retryable_error
from ._logging import logger_name
from ._types import JSONSerializable
from ._utils import (
_is_content_type_json,
_is_content_type_text,
_is_content_type_xml,
_retry_with_exp_backoff,
_retry_with_exp_backoff_async,
ignore_docs,
)
from ._utils import _retry_with_exp_backoff, _retry_with_exp_backoff_async

DEFAULT_BACKOFF_EXPONENTIAL_FACTOR = 2
DEFAULT_BACKOFF_RANDOM_FACTOR = 1
Expand Down Expand Up @@ -70,9 +65,9 @@ def _maybe_parse_response(response: httpx.Response) -> Any:
content_type = response.headers['content-type'].split(';')[0].strip()

try:
if _is_content_type_json(content_type):
if is_content_type_json(content_type):
return response.json()
elif _is_content_type_xml(content_type) or _is_content_type_text(content_type):
elif is_content_type_xml(content_type) or is_content_type_text(content_type):
return response.text
else:
return response.content
Expand Down
6 changes: 0 additions & 6 deletions src/apify_client/_types.py

This file was deleted.

Loading