Skip to content

Commit a09ca0d

Browse files
marcelklehrmarcelklehr
authored
Merge pull request #11248 from nextcloud/add/text2image
add(Text2Image): Add dev docs for TextToImage OCP and OCS API + admin docs for TextToImage
2 parents dedaae3 + e5a29f5 commit a09ca0d

File tree

5 files changed

+426
-1
lines changed

5 files changed

+426
-1
lines changed

‎admin_manual/ai/index.rst

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,8 @@ Overview of AI features
2727
"Speech-To-Text","`Whisper Speech-To-Text <https://apps.nextcloud.com/apps/stt_whisper>`_","Yellow","Yes","Yes - Whisper models by OpenAI","No","Yes"
2828
"","`OpenAI and LocalAI integration <https://apps.nextcloud.com/apps/integration_openai>`_","Yellow","Yes","Yes - Whisper models by OpenAI","No","No"
2929
"","`Replicate integration <https://apps.nextcloud.com/apps/integration_replicate>`_","Yellow","Yes","Yes - Whisper models by OpenAI","No","No"
30-
"Image generation","`OpenAI and LocalAI integration (via OpenAI API) <https://apps.nextcloud.com/apps/integration_openai>`_","Red","No","No","No","No"
30+
"Image generation","`Local Stable Diffusion <https://apps.nextcloud.com/apps/text2image_stablediffusion>`_","Yellow","Yes","Yes - StableDiffusion XL model by StabilityAI","No","Yes"
31+
"","`OpenAI and LocalAI integration (via OpenAI API) <https://apps.nextcloud.com/apps/integration_openai>`_","Red","No","No","No","No"
3132
"","`OpenAI and LocalAI integration (via LocalAI) <https://apps.nextcloud.com/apps/integration_openai>`_","Yellow","Yes","Yes - StableDiffusion models by StabilityAI","No","Yes"
3233
"","`Replicate integration <https://apps.nextcloud.com/apps/integration_replicate>`_","Yellow","Yes","Yes - StableDiffusion models by StabilityAI","No","No"
3334
"Text generation","`Local large language model (via GPT4all Falcon) <https://apps.nextcloud.com/apps/llm>`_","Green","Yes","Yes","Yes","Yes"
@@ -91,3 +92,13 @@ Implementing apps
9192

9293
* `Assistant <https://apps.nextcloud.com/apps/assistant>`_ for various tasks
9394
* `Mail <https://apps.nextcloud.com/apps/mail>`_ for summarizing mail threads (see :ref:`the Nextcloud Mail docs<mail_thread_summary>` for how to enable this)
95+
96+
97+
Image generation
98+
^^^^^^^^^^^^^^^^
99+
As you can see in the table above we have multiple apps offering Image generation capabilities. In downstream apps like the Text-to-Image helper app, users can use the image generation functionality regardless of which app implements it behind the scenes.
100+
101+
Implementing apps
102+
~~~~~~~~~~~~~~~~~
103+
104+
* `Text-to-Image Helper <https://apps.nextcloud.com/apps/stt_helper>`_ for providing a Text-to-Image smart picker

‎developer_manual/client_apis/OCS/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,3 +19,4 @@ The old documentation is still kept as it provides some additional documentation
1919
ocs-user-preferences-api
2020
ocs-translation-api
2121
ocs-textprocessing-api
22+
ocs-text2image-api

‎developer_manual/client_apis/OCS/ocs-text2image-api.rst

Lines changed: 176 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,176 @@
1+
.. _ocs-text2image-api:
2+
3+
======================
4+
OCS Text-To-Image API
5+
======================
6+
7+
.. versionadded:: 28
8+
9+
The OCS Text-To-Image API allows you to run image generation tasks implemented by apps using :ref:`the backend Text-To-Image API<text2image>`.
10+
11+
The base URL for all calls to this API is: *<nextcloud_base_url>/ocs/v2.php/text2image/*
12+
13+
All calls to OCS endpoints require the ``OCS-APIRequest`` header to be set to ``true``.
14+
15+
16+
Check availability
17+
------------------
18+
19+
.. versionadded:: 28
20+
21+
* Method: ``GET``
22+
* Endpoint: ``/is_available``
23+
* Response:
24+
- Status code:
25+
+ ``200 OK``
26+
- Data:
27+
28+
+----------------------+--------+---------------------------------------------------------------------------------------------------------------+
29+
| field | type | Description |
30+
+----------------------+--------+---------------------------------------------------------------------------------------------------------------+
31+
|``isAvailable`` | bool | Boolean indicating whether any Text-To-Image providers are installed |
32+
+----------------------+--------+---------------------------------------------------------------------------------------------------------------+
33+
34+
Schedule a task
35+
---------------
36+
37+
.. versionadded:: 28
38+
39+
.. note:: The endpoint is rate limited as it can be quite resource intensive. Users can make 20 requests in 2 minutes, guests only 5
40+
41+
* Method: ``POST``
42+
* Endpoint: ``/schedule``
43+
* Data:
44+
45+
+-------------------+-------------+--------------------------------------------------------------------------------+
46+
| field | type | Description |
47+
+-------------------+-------------+--------------------------------------------------------------------------------+
48+
|``input`` | string | The input text for the task |
49+
+-------------------+-------------+--------------------------------------------------------------------------------+
50+
|``numberOfImages`` | int | The number of images to generate (optional; default: 8) |
51+
+-------------------+-------------+--------------------------------------------------------------------------------+
52+
|``appId`` | string | The id of the requesting app |
53+
+-------------------+-------------+--------------------------------------------------------------------------------+
54+
|``identifier`` | string | An app-defined identifier for the task (optional) |
55+
+-------------------+-------------+--------------------------------------------------------------------------------+
56+
57+
If possible the task will be executed while the request is processed on the server, otherwise it is scheduled as a background job.
58+
59+
* Response:
60+
- Status code:
61+
+ ``200 OK``
62+
+ ``412 Precondition Failed`` - When the task type is not available currently
63+
+ ``429 Too Many Requests`` - When the rate limiting was exceeded
64+
65+
- Data:
66+
+ ``id`` - Only provided in case of ``200 OK``, the assigned task id, int
67+
+ ``input`` - Only provided in case of ``200 OK``, the task input, string
68+
+ ``status`` - Only provided in case of ``200 OK``, the current task status, int, see :ref:`the backend Text-To-Image API<text2image_statuses>`
69+
+ ``userId`` - Only provided in case of ``200 OK``, the originating userId of the task, string
70+
+ ``appId`` - Only provided in case of ``200 OK``, the originating appId of the task, string
71+
+ ``identifier`` - Only provided in case of ``200 OK``, the originating appId of the task, string
72+
+ ``numberOfImages`` - Only provided in case of ``200 OK``, the number of generated images, int
73+
+ ``completionExpectedAt`` - Only provided in case of ``200 OK``, the date and time when the result is expected to be completed as a UNIX timestamp, int
74+
+ ``message`` - Only provided when not ``200 OK``, an error message in the user's language, ready to be displayed
75+
76+
Fetch a task by ID
77+
------------------
78+
79+
.. versionadded:: 28
80+
81+
.. note:: The endpoint is rate limited as it can be quite resource intensive. Users can make 20 requests in 2 minutes, guests only 5
82+
83+
* Method: ``POST``
84+
* Endpoint: ``/task/{id}``
85+
86+
* Response:
87+
- Status code:
88+
+ ``200 OK``
89+
+ ``404 Not Found`` - When the task could not be found
90+
91+
- Data:
92+
+ ``id`` - Only provided in case of ``200 OK``, the assigned task id, int
93+
+ ``input`` - Only provided in case of ``200 OK``, the task input, string
94+
+ ``status`` - Only provided in case of ``200 OK``, the current task status, int, see :ref:`the backend Text-To-Image API<text2image_statuses>`
95+
+ ``userId`` - Only provided in case of ``200 OK``, the originating userId of the task, string
96+
+ ``appId`` - Only provided in case of ``200 OK``, the originating appId of the task, string
97+
+ ``identifier`` - Only provided in case of ``200 OK``, the originating appId of the task, string
98+
+ ``numberOfImages`` - Only provided in case of ``200 OK``, the number of generated images, int
99+
+ ``completionExpectedAt`` - Only provided in case of ``200 OK``, the date and time when the result is expected to be completed as a UNIX timestamp, int
100+
+ ``message`` - Only provided when not ``200 OK``, an error message in the user's language, ready to be displayed
101+
102+
Fetch a result image
103+
--------------------
104+
105+
.. versionadded:: 28
106+
107+
* Method: ``POST``
108+
* Endpoint: ``/task/{id}/image/{index}``
109+
* ``index``: The index of the image, starting at 0
110+
111+
* Response:
112+
- Status code:
113+
+ ``200 OK``
114+
+ ``404 Not Found`` - When the task could not be found, isn't successful, isn't completed yet, or the index is out of bounds
115+
116+
- Data: The raw image data
117+
118+
Delete a task
119+
-------------
120+
121+
.. versionadded:: 28
122+
123+
* Method: ``DELETE``
124+
* Endpoint: ``/task/{id}``
125+
126+
* Response:
127+
- Status code:
128+
+ ``200 OK``
129+
+ ``404 Not Found`` - When the task could not be found
130+
131+
- Data:
132+
+ ``id`` - Only provided in case of ``200 OK``, the assigned task id, int
133+
+ ``input`` - Only provided in case of ``200 OK``, the task input, string
134+
+ ``status`` - Only provided in case of ``200 OK``, the current task status, int, see :ref:`the backend Text-To-Image API<text2image_statuses>`
135+
+ ``userId`` - Only provided in case of ``200 OK``, the originating userId of the task, string
136+
+ ``appId`` - Only provided in case of ``200 OK``, the originating appId of the task, string
137+
+ ``identifier`` - Only provided in case of ``200 OK``, the originating appId of the task, string
138+
+ ``numberOfImages`` - Only provided in case of ``200 OK``, the number of generated images, int
139+
+ ``completionExpectedAt`` - Only provided in case of ``200 OK``, the date and time when the result is expected to be completed as a UNIX timestamp, int
140+
+ ``message`` - Only provided when not ``200 OK``, an error message in the user's language, ready to be displayed
141+
142+
List tasks by App
143+
------------------
144+
145+
.. versionadded:: 28
146+
147+
.. note:: The endpoint is rate limited as it can be quite resource intensive. Guests can only do 5 requests within 2 minutes
148+
149+
* Method: ``DELETE``
150+
* Endpoint: ``/tasks/app/{appId}``
151+
* Data:
152+
153+
+-------------------+-------------+--------------------------------------------------------------------------------+
154+
| field | type | Description |
155+
+-------------------+-------------+--------------------------------------------------------------------------------+
156+
|``appId`` | string | The id of the requesting app |
157+
+-------------------+-------------+--------------------------------------------------------------------------------+
158+
|``identifier`` | string | An app-defined identifier for the task (optional) |
159+
+-------------------+-------------+--------------------------------------------------------------------------------+
160+
161+
* Response:
162+
- Status code:
163+
+ ``200 OK``
164+
+ ``404 Not Found`` - When the task could not be found
165+
166+
- Data:
167+
+ Only provided in case of ``200 OK``, an array of objects:
168+
+ ``id`` - the assigned task id, int
169+
+ ``input`` - the task input, string
170+
+ ``status`` - the current task status, int, see :ref:`the backend Text-To-Image API<text2image_statuses>`
171+
+ ``userId`` - the originating userId of the task, string
172+
+ ``appId`` - the originating appId of the task, string
173+
+ ``identifier`` - the originating appId of the task, string
174+
+ ``numberOfImages`` - the number of generated images, int
175+
+ ``completionExpectedAt`` - the date and time when the result is expected to be completed as a UNIX timestamp, int
176+
+ ``message`` - Only provided when not ``200 OK``, an error message in the user's language, ready to be displayed

‎developer_manual/digging_deeper/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ Digging deeper
2727
talk
2828
translation
2929
text_processing
30+
text2image
3031
two-factor-provider
3132
users
3233
dashboard

0 commit comments

Comments
 (0)