Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: EDA on pipeline Tables #834

Merged
merged 125 commits into from
Feb 13, 2024
Merged

feat: EDA on pipeline Tables #834

merged 125 commits into from
Feb 13, 2024

Conversation

SmiteDeluxe
Copy link
Contributor

@SmiteDeluxe SmiteDeluxe commented Jan 26, 2024

Summary of Changes

  • MVP of EDA analysis through the VS Code extension
  • Working
    • Table viewing of pipeline extracted data
    • Column reodering
    • Column resizing
    • Window resizing with table adapting
    • Selecting Columns
    • Selecting Rows
    • Column header context menu to select, and unselect columns
    • Sidebar content: Table name and Row count
    • Sidebar resizing and table realigning
    • Profiling drop down with just sample text
    • Id/Row number column
    • Displaying of large data through dynamically loaded rows
    • Loading indications and proper presentation as a window of VS Code
    • Multiple exploration windows at same time and reordering them in vscode
    • returning to a running exploration if selecting same placeholder again

To start

  1. Have a pipeline with an imported table, like having this stub in directory:
package a

class Table {
    @Impure(
        [
            ImpurityReason.Other
        ]
    )
    static fun from_csv_file(name: String) -> table: Table
}

@Impure(
    [
        ImpurityReason.Other
    ]
)
@PythonCall("from safeds.data.tabular.containers import Table")
fun __import_Table()

and then this pipeline, while having the relevant CSVs stored in proper locations (can get from Kaggle, change paths accordingly):

package a

pipeline mainpipeline {
    __import_Table();
    val i = 0;
    val i2 = i + i + 3;
    val i3 = "abc";
    val i4 = true;
    val i5 = -1.2491891891;
    val Titanic23 = Table.from_csv_file("/home/jonas/titanic.csv"); // pfad
    val Cerea25l = Table.from_csv_file("/home/jonas/cereal.csv"); // pfad
    val Netflix23 = Table.from_csv_file("/home/jonas/netflix_titles.csv"); // pfad
    val Tweets123 = Table.from_csv_file("/home/jonas/tweets.csv"); // pfad
}
  1. Right click on table variable and select "Explore Table"
  2. If the selected placeholder was valid the table will show in a new window after loading

Other way:

  • Press "ctrl + shift + p" (for vscode commands) and start the EDA command, the one from right click that works if your cursor is in right position is "EDA: Explore Table"

WinPlay02 and others added 30 commits November 2, 2023 15:41
### Summary of Changes
Copied over the basics of the previous tech demo webview in svelte of the EDA tech demo in my repo
### Summary of Changes
- Ignoring configs
- allowing console logs
- allowing default exports (needed for good svelte/vite configs)
- include ts checking for shared types folder
### Summary of Changes
Added nodemon package and command to watch eda package for changes and build on change, so that extension gets new webview
### Summary of Changes
- added commands to show eda
- added Panel logic for that eda, which is using the eda packages built script
- pythonServer.ts now exporting port so that vscode can hand it to webview
- tsconfig allowing now import of ts extensions, since clashed with another linter rule
feat: use custom output channel
…rver

# Conflicts:
#	packages/safe-ds-vscode/package.json
### Summary of Changes
- selectedText now called tableIdentifier
- webview store only saving currentState not all states, way less complex and error prone & defaultState property of State to avoid initial update of global state to default
- Debugger now as new component to comment in and out when needed
- eda panel cleanup & now webview listener in constructor, fixing n + 1 execution of setting global state
- New method to reset global state
feat: document message types

misc: adapt to changes made to runner
fix: finding next free port was broken
mainly: table view design and functionalities & state/table type definitions
feat: link to source-file in stacktrace + reverse stacktrace to have the error at the top

feat: do not store execution context in global variable (would be overwritten)

docs: more docs
…rver

# Conflicts:
#	package-lock.json
#	packages/safe-ds-vscode/package.json
@lars-reimann lars-reimann force-pushed the eda branch 3 times, most recently from f0bc9cf to 8b1c318 Compare February 13, 2024 18:29
@lars-reimann lars-reimann force-pushed the eda branch 4 times, most recently from c7242d3 to b55314b Compare February 13, 2024 19:04
@lars-reimann
Copy link
Member

lars-reimann commented Feb 13, 2024

@SmiteDeluxe ESLint and Prettier should now also work for svelte files. Kindly check whether the functionality of the application is unaffected by my changes and fix the two remaining ESLint errors (the marked code looks buggy). Then this is good to go.

/github/workspace/packages/safe-ds-eda/src/components/TableView.svelte
21:13 error 'minTableWidth' is already declared in the upper scope on line 14 column 9 @typescript-eslint/no-shadow
30:13 error 'minTableWidth' is assigned a value but never used @typescript-eslint/no-unused-vars

@SmiteDeluxe
Copy link
Contributor Author

SmiteDeluxe commented Feb 13, 2024

@SmiteDeluxe ESLint and Prettier should now also work for svelte files. Kindly check whether the functionality of the application is unaffected by my changes and fix the two remaining ESLint errors (the marked code looks buggy). Then this is good to go.

/github/workspace/packages/safe-ds-eda/src/components/TableView.svelte
21:13 error 'minTableWidth' is already declared in the upper scope on line 14 column 9 @typescript-eslint/no-shadow
30:13 error 'minTableWidth' is assigned a value but never used @typescript-eslint/no-unused-vars

@lars-reimann Have fixed now. But now I seem to run into the issue again where the extension is not started properly on launch. There is no sds specific highlighting (as seen in screenshot) and under settings there is also no Safe-DS extension settings. This was caused for me by this commit/merge: 703e27b and therin the line 181 change of the vscode engine from "^1.85.0" to "^1.86.0". Let me know if it's the same for you. I just checked out the commit, ran "npm i" and then "npm run langium:generate"

image

If I revert that and then test it seems good to me though.

@lars-reimann
Copy link
Member

Try updating your VS Code to version 1.86.0. Once we have a stable release, we won't continually upgrade the minimum VS Code engine anymore.

@SmiteDeluxe
Copy link
Contributor Author

Try updating your VS Code to version 1.86.0. Once we have a stable release, we won't continually upgrade the minimum VS Code engine anymore.

That worked and makes a lot of sense. We are good to merge then.

Copy link
Member

@lars-reimann lars-reimann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic effort, thanks a lot for your contribution! 🚀

@lars-reimann lars-reimann merged commit f42c9aa into main Feb 13, 2024
7 checks passed
@lars-reimann lars-reimann deleted the eda branch February 13, 2024 20:30
lars-reimann pushed a commit that referenced this pull request Feb 26, 2024
## [0.8.0](v0.7.0...v0.8.0) (2024-02-26)

### Features

* check the runner version to be compatible ([#888](#888)) ([83378a3](83378a3)), closes [#880](#880)
* check type parameter bounds for default values and named types ([#919](#919)) ([7003ea6](7003ea6)), closes [#614](#614)
* clamp default values of parameter types to upper bound ([#921](#921)) ([76ad869](76ad869))
* compute highest common subtype ([#901](#901)) ([5630a9f](5630a9f)), closes [#860](#860)
* compute type parameters for calls ([#920](#920)) ([35dc826](35dc826)), closes [#861](#861)
* consider nullability of upper type parameter bound in various checks ([#892](#892)) ([940515a](940515a))
* constraints for segments ([#911](#911)) ([e57f886](e57f886)), closes [#903](#903)
* EDA on pipeline Tables ([#834](#834)) ([f42c9aa](f42c9aa))
* error if parent type is nullable ([#891](#891)) ([add650d](add650d))
* escape characters in string conversion of string constants ([#908](#908)) ([72a9c3c](72a9c3c)), closes [#904](#904)
* format upper bound of type parameters ([#898](#898)) ([9d6ce28](9d6ce28))
* handle invariant/covariant type parameters when computing lowest common supertype ([#868](#868)) ([4d6cb4e](4d6cb4e)), closes [#860](#860) [#861](#861)
* improve handling of subclasses of lists/maps ([#890](#890)) ([bb0c94b](bb0c94b))
* improvements to constraints ([#896](#896)) ([b81bef9](b81bef9)), closes [#18](#18) [#860](#860) [#18](#18)
* include enum name in string conversion of enum variant types ([#907](#907)) ([1f6502e](1f6502e)), closes [#902](#902)
* scoping for member accesses if receiver has type parameter type ([#889](#889)) ([1277bd1](1277bd1))
* shorter message if runner is started but files have errors ([#912](#912)) ([b611b44](b611b44)), closes [#910](#910)
* simplify union types containing type parameter types disregarding entry order ([#894](#894)) ([cf6e77e](cf6e77e))
* singular type is its own lowest common supertype ([#893](#893)) ([79d611d](79d611d))
* substitute type parameter when type checking calls ([#923](#923)) ([2e09306](2e09306)), closes [#915](#915)
* substitute type parameters when checking overridden members ([#922](#922)) ([0e657cf](0e657cf)), closes [#917](#917)
* update to `safe-ds-runner` v0.7.0 ([#906](#906)) ([070f406](070f406))
* use bounds of type system where possible ([#899](#899)) ([cf92762](cf92762))

### Bug Fixes

* lenient type checking involving type parameter types ([#916](#916)) ([b9d3641](b9d3641)), closes [#915](#915) [#915](#915)
* runner startup silent failure ([#914](#914)) ([8812944](8812944)), closes [#909](#909)
* save dirty files before running pipeline ([#918](#918)) ([4302ca6](4302ca6))
* simplification of union types ([#897](#897)) ([4c577a3](4c577a3))
@lars-reimann
Copy link
Member

🎉 This PR is included in version 0.8.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@lars-reimann lars-reimann added the released Included in a release label Feb 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
released Included in a release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants