Create a new data connector for Notion

[ajjimeno]

Create a data connector that:

• fetches data from Notion.
• stores the content locally (at least temporarily for processing), and runs them through unstructured.partition.auto.
• inspiration for processing is available here

See Adding Data Connectors for details on how to get started. Make sure to include a link to this issue when submitting a PR.

Definition of Done

• The checklist has been completed.
• The connector is able to process entries from Notion using the API.
• The connector is able to process a single entry.
• The connector is able to process several entries, with an option to process them recursively.
• The connector can accept any credentials, if necessary.
• The connector should be able of processing documents through unstructured.partition.auto.

[HAKSOAT]

Hi @ajjimeno I'm looking to pick this up.

What does an entry refer to here? Does it mean a Notion page?

[ajjimeno]

Hi @HAKSOAT, thank you for the interest. Yes, the idea to being able of recovering a Notion page and, if there are nested ones recover them as well. I think the term processing might be confusing but the idea is to connect to Notion, recover the page, nested pages and possible additional attached content. The content recovered should be processed by unstructured.partition.auto.

[HAKSOAT]

Thanks for clarifying. I picked up the Google Drive connector enhancement. I hope to come back to do this when that is done.

[HAKSOAT]

Just putting an update here that I am now taking a look at this task.

[ajjimeno]

Great! Thanks Habeeb!

…

On Mon, Mar 6, 2023 at 6:34 AM Habeeb Shopeju ***@***.***> wrote: Just putting an update here that I am now taking a look at this task. — Reply to this email directly, view it on GitHub <#260 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA6BZDNYINRUGM33Z35G7ITW2TTF3ANCNFSM6AAAAAAVFE3CIE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[HAKSOAT]

Thought to share my findings after exploring the Notion API here. I didn't proceed with this because it felt a bit complex to work with Notion's API, especially in a short timeframe.

They use the concept of "blocks". Hence, multiple "blocks" per page with the possibility of nested blocks (say in the case of bullet points or toggles). I believe an approach that builds an actual page first (maybe a markdown file as the result) before passing through will be needed to make this work.

I think the following links will be good places to start from when working on this:

https://developers.notion.com/reference/block
https://developers.notion.com/reference/get-block-children
https://developers.notion.com/reference/property-value-object

[ajjimeno]

Thanks for the pointers. Interesting the way it works.

…

On Sat, Mar 11, 2023 at 8:40 AM Habeeb Shopeju ***@***.***> wrote: Thought to update my thoughts after exploring the Notion API here. I didn't proceed with this because it felt a bit complex to work with Notion's API. They use the concept of "blocks". Hence, multiple "blocks" per page with the possibility of nested blocks (say in the case of bullet points or toggles). I believe an approach that builds an actual page first (maybe a markdown file as the result) before passing through will be needed to make this work. I think the following links will be good places to start from when working on this: https://developers.notion.com/reference/block https://developers.notion.com/reference/get-block-children https://developers.notion.com/reference/property-value-object — Reply to this email directly, view it on GitHub <#260 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA6BZDPHI7H5SGZEQDXNFV3W3ONWRANCNFSM6AAAAAAVFE3CIE> . You are receiving this because you were mentioned.Message ID: ***@***.***>