Skip to content

Commit

Permalink
Update with scraper data and ui fix
Browse files Browse the repository at this point in the history
  • Loading branch information
isaidnocookies committed Mar 19, 2020
1 parent 95a8890 commit 4de6200
Show file tree
Hide file tree
Showing 5 changed files with 66 additions and 4 deletions.
8 changes: 6 additions & 2 deletions _sidebar.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,14 @@
* Github Browser
* [Setup](/docs/primitive-setup.md)

* Private Git Scraper
* Private Scraper Setup
* [Requirements](/docs/private/requirements.md)
* [Setup](/docs/private/private-setup.md)
* [Docker Setup](/docs/private/docker-setup.md)
* [Running Scraper in Docker](/docs/private/docker-usage.md)
* [Environment File](/docs/private/environment-file.md)
* [Volume](/docs/private/volume.md)
* [Volume](/docs/private/volume.md)

* Private Scraper Usage
* [Usage](/docs/private/scraper-usage.md)
* [Troubleshooting](/docs/private/scraper-troubleshooting.md)
13 changes: 13 additions & 0 deletions docs/private/scraper-troubleshooting.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Troubleshooting

| Issue | Possible Solution |
| -------------------- | ----------------- |
| Preview is blank | The preview might be blank if the scrape was interrupted, or an error occured. This could be from the process being rate limited, the URL being incorrect, or network connectivity. Please check the URL and ensure it is in the correct format, make sure that the rate limits on your git service are appropriately set, and confirm that the scraper can access the git service |
| Some assets are not present (Repository links result in 404) | See above issue. This could also be caused by rate limiting issues, incorrect URLS, and/or network connectivity |
| Assets are valid but source code is unavailable in the client. | The base url could be incorrectly set in the scrapers configuration. Check to see that the 'url' value within the 'Package Data Url' is valid and points to the appropriate resource. |
| Client is unavailable to view source code and raw URLS are valid | Ensure that the client is properly authenticating to the git service. Source code is streamed from the git service to the Primitive client and is not handled by the scraper. Please check the credentials within the client to ensure proper authentication is occuring. |
| Raw URLS are malformed | Please check the environment file used to configure the scraper during setup. The 'provider' and 'base url' values should be validated to ensure the scraper is properly configured.

## Contact Us

If issues are unabled to be resolved, please contact us at **[email protected]**.
45 changes: 45 additions & 0 deletions docs/private/scraper-usage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Private Scraper Usage

## Scraper Admin Interface

The scraper interface can be broken down into two main functions: scraping new repositories, and managing previously scraped ones.

#### Scraping New Repositories

At the top of the admin panel, there is a text field labeled 'Repository URL' with a 'Scrape Repo!' button. To scrape a new repository within your private infrastructure, input the http/https clone url in the text field and press the scrape button. It's that easy! There are a couple of notes here...
- The http/https clone url should not include your username. The url should be in the format **[baseurl]/[organization/owner]/[project name]**.
- The base url can include the protocl (http/https) and the associated port (7990/3000/etc).

#### Managing Existing Repositories

Once a repository is scraped and ready for the client, you will see a "complete" message within the status bar at the bottom of the page. On a page refresh, the newly scraped repository should appear in the Repo table.

Once in the repo table, you will have the ability to edit the metadata of the repository for display within the Primitive Client, remove the repository, requeue the repository, and view the current data saved.

##### Requeue

Requeuing removes the current cache entry and re-scrapes the repository. This can be done to update the repo in the Primitive client to the most recent version pushed to your master branch.

##### Remove

Removing the repository removes the repo data from the scraper. This removes the ability for the Primitive client to graph the codebase and the repository must be re-scraped in order to restore this ability.

##### Repository Values

The repository values include a tag, layer, and enabled field.

| Value | Description |
| ------------- |:-------------|
| **Repo** | The repository url that represents the scraped data. This should resemble the http/https clone url. There should be no username within the url and the '.git' can be removed. This is for standardization across git platforms. |
| **Tag** | The tag is the group label used to organize the repositories within the Primitive environment. Each tagged repository will be grouped based on their tag for easier navigation of the codebases |
| **Layer** | The layer is the value associated with which level the repository is placed in. The core of the Primitive structure is layer 0 and the layer number increments as you move away from the center. -1 is the default which puts it on the outer layer. |
| **Enabled** | The enabled flag allows you to remove the repository from the Primitive environment without deleting the data. A value of '0' disables the repository where a value of '1' enables it. |

#### Repository Preview

The image that you see at the top of the model is a quick preview of what the codebase structure looks like. This is used as a starting point in the visualization process and becomes the foundation of the VR structure. If there is no preview present, something may have gone wrong. Please check that the URL is correct and there is no connectivity issues.

##### Repository Links

The links included in the modal represent the data being consumed by the Primitive client. All other information pertaining to the repository are streamed into the client on demand.

2 changes: 1 addition & 1 deletion home.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<p align=center>
<img src="/_media/primitive_logo.png" height="100px">
<img src="/_media/primitive_logo.png" style="height=100px; width=auto;">
</p>

# Welcome to Primitive!
Expand Down
2 changes: 1 addition & 1 deletion index.html
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
<meta charset="UTF-8">
<title>Primitive.io</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1" />
<meta name="description" content="Description">
<meta name="description" content="Primitive documentation and informational pages.">
<meta name="viewport" content="width=device-width, user-scalable=no, initial-scale=1.0, maximum-scale=1.0, minimum-scale=1.0">
<link rel="stylesheet" href="//unpkg.com/docsify/lib/themes/vue.css">
</head>
Expand Down

0 comments on commit 4de6200

Please sign in to comment.