From 4de620070829e2ac33f5735f12f45a7750722af0 Mon Sep 17 00:00:00 2001 From: isaidnocookies Date: Thu, 19 Mar 2020 11:47:39 -0700 Subject: [PATCH] Update with scraper data and ui fix --- _sidebar.md | 8 +++-- docs/private/scraper-troubleshooting.md | 13 +++++++ docs/private/scraper-usage.md | 45 +++++++++++++++++++++++++ home.md | 2 +- index.html | 2 +- 5 files changed, 66 insertions(+), 4 deletions(-) create mode 100644 docs/private/scraper-troubleshooting.md create mode 100644 docs/private/scraper-usage.md diff --git a/_sidebar.md b/_sidebar.md index facd759..e22da27 100644 --- a/_sidebar.md +++ b/_sidebar.md @@ -4,10 +4,14 @@ * Github Browser * [Setup](/docs/primitive-setup.md) -* Private Git Scraper +* Private Scraper Setup * [Requirements](/docs/private/requirements.md) * [Setup](/docs/private/private-setup.md) * [Docker Setup](/docs/private/docker-setup.md) * [Running Scraper in Docker](/docs/private/docker-usage.md) * [Environment File](/docs/private/environment-file.md) - * [Volume](/docs/private/volume.md) \ No newline at end of file + * [Volume](/docs/private/volume.md) + +* Private Scraper Usage + * [Usage](/docs/private/scraper-usage.md) + * [Troubleshooting](/docs/private/scraper-troubleshooting.md) \ No newline at end of file diff --git a/docs/private/scraper-troubleshooting.md b/docs/private/scraper-troubleshooting.md new file mode 100644 index 0000000..134fb6f --- /dev/null +++ b/docs/private/scraper-troubleshooting.md @@ -0,0 +1,13 @@ +# Troubleshooting + +| Issue | Possible Solution | +| -------------------- | ----------------- | +| Preview is blank | The preview might be blank if the scrape was interrupted, or an error occured. This could be from the process being rate limited, the URL being incorrect, or network connectivity. Please check the URL and ensure it is in the correct format, make sure that the rate limits on your git service are appropriately set, and confirm that the scraper can access the git service | +| Some assets are not present (Repository links result in 404) | See above issue. This could also be caused by rate limiting issues, incorrect URLS, and/or network connectivity | +| Assets are valid but source code is unavailable in the client. | The base url could be incorrectly set in the scrapers configuration. Check to see that the 'url' value within the 'Package Data Url' is valid and points to the appropriate resource. | +| Client is unavailable to view source code and raw URLS are valid | Ensure that the client is properly authenticating to the git service. Source code is streamed from the git service to the Primitive client and is not handled by the scraper. Please check the credentials within the client to ensure proper authentication is occuring. | +| Raw URLS are malformed | Please check the environment file used to configure the scraper during setup. The 'provider' and 'base url' values should be validated to ensure the scraper is properly configured. + +## Contact Us + +If issues are unabled to be resolved, please contact us at **support@primitive.io**. \ No newline at end of file diff --git a/docs/private/scraper-usage.md b/docs/private/scraper-usage.md new file mode 100644 index 0000000..056f579 --- /dev/null +++ b/docs/private/scraper-usage.md @@ -0,0 +1,45 @@ +# Private Scraper Usage + +## Scraper Admin Interface + +The scraper interface can be broken down into two main functions: scraping new repositories, and managing previously scraped ones. + +#### Scraping New Repositories + +At the top of the admin panel, there is a text field labeled 'Repository URL' with a 'Scrape Repo!' button. To scrape a new repository within your private infrastructure, input the http/https clone url in the text field and press the scrape button. It's that easy! There are a couple of notes here... +- The http/https clone url should not include your username. The url should be in the format **[baseurl]/[organization/owner]/[project name]**. +- The base url can include the protocl (http/https) and the associated port (7990/3000/etc). + +#### Managing Existing Repositories + +Once a repository is scraped and ready for the client, you will see a "complete" message within the status bar at the bottom of the page. On a page refresh, the newly scraped repository should appear in the Repo table. + +Once in the repo table, you will have the ability to edit the metadata of the repository for display within the Primitive Client, remove the repository, requeue the repository, and view the current data saved. + +##### Requeue + +Requeuing removes the current cache entry and re-scrapes the repository. This can be done to update the repo in the Primitive client to the most recent version pushed to your master branch. + +##### Remove + +Removing the repository removes the repo data from the scraper. This removes the ability for the Primitive client to graph the codebase and the repository must be re-scraped in order to restore this ability. + +##### Repository Values + +The repository values include a tag, layer, and enabled field. + +| Value | Description | +| ------------- |:-------------| +| **Repo** | The repository url that represents the scraped data. This should resemble the http/https clone url. There should be no username within the url and the '.git' can be removed. This is for standardization across git platforms. | +| **Tag** | The tag is the group label used to organize the repositories within the Primitive environment. Each tagged repository will be grouped based on their tag for easier navigation of the codebases | +| **Layer** | The layer is the value associated with which level the repository is placed in. The core of the Primitive structure is layer 0 and the layer number increments as you move away from the center. -1 is the default which puts it on the outer layer. | +| **Enabled** | The enabled flag allows you to remove the repository from the Primitive environment without deleting the data. A value of '0' disables the repository where a value of '1' enables it. | + +#### Repository Preview + +The image that you see at the top of the model is a quick preview of what the codebase structure looks like. This is used as a starting point in the visualization process and becomes the foundation of the VR structure. If there is no preview present, something may have gone wrong. Please check that the URL is correct and there is no connectivity issues. + +##### Repository Links + +The links included in the modal represent the data being consumed by the Primitive client. All other information pertaining to the repository are streamed into the client on demand. + diff --git a/home.md b/home.md index 07ed98e..68db726 100644 --- a/home.md +++ b/home.md @@ -1,5 +1,5 @@

- +

# Welcome to Primitive! diff --git a/index.html b/index.html index a572be3..f1db585 100644 --- a/index.html +++ b/index.html @@ -4,7 +4,7 @@ Primitive.io - +