Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New functions of clickhouse. Direction of infi #138

Open
emakarov opened this issue Dec 14, 2019 · 3 comments
Open

New functions of clickhouse. Direction of infi #138

emakarov opened this issue Dec 14, 2019 · 3 comments

Comments

@emakarov
Copy link
Contributor

Hi @ishirav

ClickHouse is moving forward, and in this issue I wanted to understand what is your vision on keeping with CH changes.

In particular, new features of CH (implemented and upcoming) are related to:

  • User permissions (GRANT, etc..)
  • RBAC control
  • more functions, including ML (simple regressions for now but can be much more in future). I've read that work on functions is in progress
  • Constraints and templates for data insertions
  • ClickHouse internal tables for statistics and performance/monitoring

And many other features

What is your vision on infi.clickhouse_orm.
Should support of various such features be incorporated into it or no?
Is there any criteria what should be included into ORM, what should not be?

@ishirav
Copy link
Contributor

ishirav commented Dec 15, 2019

Theoretically, the ORM should cover as many database features as possible. Realistically, the ClickHouse developers are adding features a lot faster than I can keep up with, given the limited time I have for working on this project. Of course, I'm always open to pull requests.

My current plan is to release version 2.0, which will support functions and expressions everywhere - a big change in terms of the ORM's expressiveness. It will also support IPv4 and IPv6 fields, but probably no other new database features (since I don't want to delay the release, better to get it out there). Version 2.0 will no longer support Python 2.7, but should be otherwise backwards-compatible.

After v2.0 I'll work on v2.1+ which will cover new database features. The ones I had in mind are:

  • New MergeTree features
  • Column comments
  • TTL expressions
  • Delete rows
  • Update rows

Perhaps also:

  • Column constraints
  • ORDER BY ... WITH FILL

Note that this is a very tentative list, and I'm open to discussion. Generally I implement features by public demand, ease of implementation, having a good fit with the ORM, and apparent usefulness.

As for other database features you mentioned:

  • Permissions and RBAC: I don't see any documentation about these, so I don't know yet.
  • Templates for insertion: I'll need to look into this, although it seems less useful when working in a Python environment since you can just parse the data using Python and convert it into ORM instances before inserting it.
  • Internal tables: it seems to me that the structure of these tables changes often between database versions, so it might be problematic to create models for them. Perhaps I could make something dynamic that builds a model for system tables based on their definition, e.g. you would call database.getSystemModel("part_log") and get a model class that you can use for queries. That might be cool.

@emakarov
Copy link
Contributor Author

emakarov commented Dec 16, 2019

Thanks for such detailed response!

Permissions and RBAC: I don't see any documentation about these, so I don't know yet.

Yes, this is still in progress but they want to release it in December or early 2020

@ishirav
Copy link
Contributor

ishirav commented Jul 16, 2020

Some of the features listed above were added to v2.1.0:

  • Support for model constraints
  • Support for data skipping indexes
  • Support for mutations: QuerySet.update and QuerySet.delete

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants