Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug/Extensions .mdx and .markdown not supported #3670

Closed
butasebi opened this issue Sep 27, 2024 · 3 comments
Closed

bug/Extensions .mdx and .markdown not supported #3670

butasebi opened this issue Sep 27, 2024 · 3 comments
Assignees
Labels
bug Something isn't working filetype Related to file-type detection for partitioning dispatch

Comments

@butasebi
Copy link

Describe the bug
The extensions .mdx and .markdown are being transformed to FileType.UNK when being passed to unstructured.file_utils.filetype.detect_filetype

To Reproduce
from unstructured.file_utils.filetype import detect_filetype
print(detect_filetype("file.mdx"))
print(detect_filetype("file.markdown"))

Expected behavior
The expected behavior should be either to have them go into a FileType.MDX respectively FileType.MARKDOWN (just like XLS XLSX) or at least have them be FileType.MD

@butasebi butasebi added the bug Something isn't working label Sep 27, 2024
@scanny scanny added the filetype Related to file-type detection for partitioning dispatch label Dec 16, 2024
@scanny
Copy link
Collaborator

scanny commented Dec 16, 2024

@butasebi I don't think the .mdx is going to work out because that format cannot be parsed by the python-markdown package we use. It's really a hybrid format.

But a file with .markdown should identify as such. I'm currently getting FileType.TXT for that case, where the same file with a .md extension is correctly identified. I'll have a closer look at why that is.

@scanny
Copy link
Collaborator

scanny commented Dec 16, 2024

Yep, okay, found that one. I'll get that into the next release.

@scanny scanny self-assigned this Dec 16, 2024
@scanny
Copy link
Collaborator

scanny commented Dec 18, 2024

Fixed in #3828.

@scanny scanny closed this as completed Dec 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working filetype Related to file-type detection for partitioning dispatch
Projects
None yet
Development

No branches or pull requests

2 participants