Skip to content

Commit

Permalink
Update documentation (#713)
Browse files Browse the repository at this point in the history 
Co-authored-by: daniil-quix <[email protected]>
  • Loading branch information
@github-actions @daniil-quix
github-actions[bot] and daniil-quix authored Jan 15, 2025
1 parent 63272b9 commit 1c43924
Show file tree
Hide file tree
Showing 3 changed files with 72 additions and 72 deletions.
30 changes: 15 additions & 15 deletions docs/api-reference/application.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
class Application()
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L75)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L78)

The main Application class.

Expand Down Expand Up @@ -85,7 +85,7 @@ def __init__(broker_address: Optional[Union[str, ConnectionConfig]] = None,
processing_guarantee: ProcessingGuarantee = "at-least-once")
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L113)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L116)


<br>
Expand Down Expand Up @@ -174,7 +174,7 @@ instead of the default one.
def Quix(cls, *args, **kwargs)
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L352)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L358)

RAISES EXCEPTION: DEPRECATED.

Expand All @@ -197,7 +197,7 @@ def topic(name: str,
timestamp_extractor: Optional[TimestampExtractor] = None) -> Topic
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L384)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L390)

Create a topic definition.

Expand Down Expand Up @@ -279,7 +279,7 @@ def dataframe(topic: Optional[Topic] = None,
source: Optional[BaseSource] = None) -> StreamingDataFrame
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L464)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L470)

A simple helper method that generates a `StreamingDataFrame`, which is used

Expand Down Expand Up @@ -335,7 +335,7 @@ to be used as an input topic.
def stop(fail: bool = False)
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L520)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L526)

Stop the internal poll loop and the message processing.

Expand All @@ -362,7 +362,7 @@ to unhandled exception, and it shouldn't commit the current checkpoint.
def get_producer() -> Producer
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L565)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L571)

Create and return a pre-configured Producer instance.
The Producer is initialized with params passed to Application.
Expand Down Expand Up @@ -397,7 +397,7 @@ with app.get_producer() as producer:
def get_consumer(auto_commit_enable: bool = True) -> Consumer
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L613)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L626)

Create and return a pre-configured Consumer instance.

Expand Down Expand Up @@ -454,7 +454,7 @@ with app.get_consumer() as consumer:
def clear_state()
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L663)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L676)

Clear the state of the application.

Expand All @@ -468,7 +468,7 @@ Clear the state of the application.
def add_source(source: BaseSource, topic: Optional[Topic] = None) -> Topic
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L669)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L682)

Add a source to the application.

Expand All @@ -495,7 +495,7 @@ Note: the names of default topics are prefixed with "source__".
def run(dataframe: Optional[StreamingDataFrame] = None)
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L700)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L715)

Start processing data from Kafka using provided `StreamingDataFrame`

Expand Down Expand Up @@ -531,7 +531,7 @@ app.run()
def setup_topics()
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L824)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L839)

Validate and create the topics

Expand All @@ -543,7 +543,7 @@ Validate and create the topics
class ApplicationConfig(BaseSettings)
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L1016)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L1027)

Immutable object holding the application configuration

Expand All @@ -566,7 +566,7 @@ def settings_customise_sources(
) -> Tuple[PydanticBaseSettingsSource, ...]
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L1051)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L1062)

Included to ignore reading/setting values from the environment

Expand All @@ -580,7 +580,7 @@ Included to ignore reading/setting values from the environment
def copy(**kwargs) -> Self
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L1064)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/app.py#L1075)

Update the application config and return a copy

42 changes: 21 additions & 21 deletions docs/api-reference/kafka.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -229,7 +229,7 @@ def poll(
) -> Optional[RawConfluentKafkaMessageProto]
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L146)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L145)

Consumes a single message, calls callbacks and returns events.

Expand Down Expand Up @@ -267,7 +267,7 @@ event or callback. None or -1 is infinite. Default: None.
def unsubscribe()
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L251)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L250)

Remove current subscription.

Expand All @@ -287,7 +287,7 @@ def store_offsets(message: Optional[Message] = None,
offsets: Optional[List[TopicPartition]] = None)
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L260)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L259)

Store offsets for a message or a list of offsets.

Expand Down Expand Up @@ -320,7 +320,7 @@ def commit(message: Optional[Message] = None,
asynchronous: bool = True) -> Optional[List[TopicPartition]]
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L291)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L290)

Commit a message or a list of offsets.

Expand Down Expand Up @@ -358,7 +358,7 @@ def committed(partitions: List[TopicPartition],
timeout: Optional[float] = None) -> List[TopicPartition]
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L332)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L331)

Retrieve committed offsets for the specified partitions.

Expand Down Expand Up @@ -393,7 +393,7 @@ def get_watermark_offsets(partition: TopicPartition,
cached: bool = False) -> Tuple[int, int]
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L350)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L349)

Retrieve low and high offsets for the specified partition.

Expand Down Expand Up @@ -432,7 +432,7 @@ def list_topics(topic: Optional[str] = None,
timeout: Optional[float] = None) -> ClusterMetadata
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L376)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L375)

Request metadata from the cluster.

Expand Down Expand Up @@ -464,7 +464,7 @@ None or -1 is infinite. Default: None
def memberid() -> Optional[str]
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L397)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L396)

Return this client's broker-assigned group member id.

Expand Down Expand Up @@ -492,7 +492,7 @@ def offsets_for_times(partitions: List[TopicPartition],
timeout: Optional[float] = None) -> List[TopicPartition]
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L410)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L409)

Look up offsets by timestamp for the specified partitions.

Expand Down Expand Up @@ -531,7 +531,7 @@ None or -1 is infinite. Default: None
def pause(partitions: List[TopicPartition])
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L436)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L435)

Pause consumption for the provided list of partitions.

Expand Down Expand Up @@ -559,7 +559,7 @@ Does NOT affect the result of `Consumer.assignment()`.
def resume(partitions: List[TopicPartition])
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L449)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L448)

Resume consumption for the provided list of partitions.

Expand All @@ -583,7 +583,7 @@ Resume consumption for the provided list of partitions.
def position(partitions: List[TopicPartition]) -> List[TopicPartition]
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L459)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L458)

Retrieve current positions (offsets) for the specified partitions.

Expand Down Expand Up @@ -616,7 +616,7 @@ the last consumed message + 1.
def seek(partition: TopicPartition)
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L473)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L472)

Set consume position for partition to offset.

Expand Down Expand Up @@ -648,7 +648,7 @@ pass the offset in an `assign()` call.
def assignment() -> List[TopicPartition]
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L490)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L489)

Returns the current partition assignment.

Expand All @@ -673,7 +673,7 @@ Returns the current partition assignment.
def set_sasl_credentials(username: str, password: str)
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L503)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L502)

Sets the SASL credentials used for this client.

Expand All @@ -700,7 +700,7 @@ This method is applicable only to SASL PLAIN and SCRAM mechanisms.
def incremental_assign(partitions: List[TopicPartition])
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L517)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L516)

Assign new partitions.

Expand All @@ -726,7 +726,7 @@ Any additional partitions besides the ones passed during the `Consumer`
def assign(partitions: List[TopicPartition])
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L531)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L530)

Set the consumer partition assignment to the provided list of `TopicPartition` and start consuming.

Expand All @@ -751,7 +751,7 @@ Set the consumer partition assignment to the provided list of `TopicPartition` a
def incremental_unassign(partitions: List[TopicPartition])
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L541)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L540)

Revoke partitions.

Expand All @@ -773,7 +773,7 @@ Can be called outside an on_revoke callback.
def unassign()
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L551)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L550)

Removes the current partition assignment and stops consuming.

Expand All @@ -792,7 +792,7 @@ Removes the current partition assignment and stops consuming.
def close()
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L560)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L559)

Close down and terminate the Kafka Consumer.

Expand All @@ -815,7 +815,7 @@ see `poll()` for more info.
def consumer_group_metadata() -> GroupMetadata
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L577)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/kafka/consumer.py#L576)

Used by the producer during consumer offset sending for an EOS transaction.

Loading

0 comments on commit 1c43924

Please sign in to comment.