Skip to content

Commit

Permalink
autonomous running in background, but more testing needed
Browse files Browse the repository at this point in the history
  • Loading branch information
sahil-lalani committed Nov 13, 2024
1 parent 9ade9ab commit 851c223
Show file tree
Hide file tree
Showing 19 changed files with 471 additions and 420 deletions.
15 changes: 6 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Surfer: Export your personal data in one click

# 🆕 **RECENT UPDATES (10/28/24): Surfer is evolving! We're currently working on a [new protocol](https://surferprotocol.org) to document personal data exporting. For the latest updates, visit our [project page](https://surferprotocol.org) or the [GitHub repo](https://github.com/surfer-org/protocol).** 🆕
# 🆕 **RECENT UPDATES (10/28/24): Surfer is evolving! We're currently working on an [open-source framework](https://surferprotocol.org) to get and do stuff with your personal data. For the latest updates, visit our [project page](https://surferprotocol.org) or the [GitHub repo](https://github.com/surfer-org/protocol).** 🆕

[![Contributors][contributors-shield]][contributors-url]
[![Forks][forks-shield]][forks-url]
Expand Down Expand Up @@ -35,17 +35,14 @@ Currently, your personal data is scattered across hundreds of platforms and the
We believe that personal data aggregation is the key to enabling truly useful, universal personal assistants.

## Currently Supported Platforms

- Twitter Posts
- iMessages (Windows only)
- Twitter Bookmarks
- LinkedIn Profile
- GitHub Repositories
- YouTube
- Notion
- ChatGPT History
- Gmail
- iMessages (coming soon!)
- LinkedIn Connections (coming soon!)
- Reddit (coming soon!)
- GitHub (coming soon!)

## How it works

Expand Down Expand Up @@ -88,10 +85,10 @@ See the [open issues](https://github.com/CEREBRUS-MAXIMUS/Surfer-Data/issues) fo

### Short-Term
- [x] Data being maintained/updated everyday
- [ ] Scheduled exports
- [x] Scheduled exports
- [ ] Obtain a code signing certificate for Windows
- [x] Replace `setTimeout` with `await` for script execution to ensure elements exist before scraping
- [ ] Implement robust error handling for the scraping process
- [ ] Iamplement robust error handling for the scraping process
- [ ] Add support for more online platforms
- [x] Add verbosity to runs

Expand Down
Binary file modified assets/icon.icns
Binary file not shown.
Binary file removed assets/icon.ico
Binary file not shown.
Binary file modified assets/icon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
50 changes: 11 additions & 39 deletions assets/icon.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
6 changes: 4 additions & 2 deletions docs/ADD_PLATFORMS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

To add support for a new platform in Surfer, follow these steps:

1. **Create a new directory**: In the `Companies` folder, create a new directory named after the company (e.g., `Companies/Salesforce`).
1. **Create a new directory**: In the `platforms` folder, create a new directory named after the company (e.g., `platforms/Salesforce`).

2. **Create the platform file**: Inside the new directory, create a JavaScript file named after the platform (e.g., `slack.js`).

Expand All @@ -12,6 +12,7 @@ To add support for a new platform in Surfer, follow these steps:
- `connectURL`: The URL for the platform's login page
- `connectSelector`: A CSS selector for an element that indicates a successful login
- `isUpdated` (optional): A boolean indicating if the platform's data is regularly updated
- `exportFrequency` (optional): The frequency at which the platform's data is exported (e.g., "daily", "weekly", "monthly")

Example JSON structure:
```json
Expand All @@ -20,7 +21,8 @@ To add support for a new platform in Surfer, follow these steps:
"description": "Exports [specific data types].",
"connectURL": "https://platform.com/login",
"connectSelector": "CSS_SELECTOR_FOR_LOGGED_IN_STATE",
"isUpdated": true
"isUpdated": true,
"exportFrequency": "daily"
}
```

Expand Down
48 changes: 48 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,7 @@
"lucide-react": "^0.396.0",
"mbox-parser": "^1.0.1",
"next-themes": "^0.3.0",
"node-schedule": "^2.1.1",
"npm": "^10.8.2",
"openai": "^4.58.2",
"os-browserify": "^0.3.0",
Expand Down Expand Up @@ -272,7 +273,7 @@
"notarize": {
"teamId": "97SB7MA2WB"
},
"icon": "assets/mac-icon.icns"
"icon": "assets/icon.icns"
},
"dmg": {
"sign": false,
Expand Down
Loading

0 comments on commit 851c223

Please sign in to comment.