Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Add repository metrics #177

Open
wants to merge 20 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
6cebdbb
First version of integrated repository metrics
steffen Feb 27, 2019
d5320c7
Add passing around nwo in query string in navigation links
steffen Feb 27, 2019
69b3577
Add loading repository specific data
steffen Feb 27, 2019
2480fd4
Add back to instance statistics link to repo view
steffen Feb 28, 2019
4495986
Add overview table to overview page
steffen Feb 28, 2019
5b33604
Add first version of creating a report for monitored repositories
steffen Mar 1, 2019
b68d0a3
Add report for GitHub api request types by count
steffen Mar 1, 2019
0d2b923
Add abstract ReportRepository class
steffen Mar 1, 2019
93c8f0f
Add monitored repositories
steffen Mar 1, 2019
06384ab
Add showing repository name in page header
steffen Mar 1, 2019
13ee2ec
Add writing repository name directly into the header to avoid flickering
steffen Mar 1, 2019
d9495fd
Add git-sizer metrics
steffen Apr 5, 2019
44b62be
Make "passing" repository to scripts work
steffen Apr 5, 2019
221fafc
Exclude internal api requests from api request types data
steffen Apr 5, 2019
a75e34a
Add repository metrics support for non-remote runs
steffen Apr 17, 2019
654b75b
Add repository Git traffic metrics
steffen Apr 17, 2019
7008ec9
Consolidate git-sizer report and include all quantity values
steffen Apr 17, 2019
189d46c
Tweaked charts to focus on actionable metrics
steffen Apr 18, 2019
c1b1a8a
Make updater scripts executable
steffen May 9, 2019
eff5fca
Fix issue with relative dataURLs by only supporting absolute dataURLs
steffen Aug 18, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/_config.yml
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
dataURL: https://autodesk.github.io/hubble/demo-data
dataURL: https://steffen.github.io/hubble/demo-data
exclude: ['spec', 'node_modules', 'README.md', '.gitignore', 'package.json', 'package-lock.json']
2 changes: 2 additions & 0 deletions docs/_data/navigation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@
url: "/repos-activity"
- title: "Total"
url: "/repos-total"
- title: "Monitored"
url: "/repos-monitored"
- title: "Organizations"
url: "/orgs-activity"
subnavigation:
Expand Down
14 changes: 14 additions & 0 deletions docs/_data/repository_navigation.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
- title: "Overview"
url: "/repository/"
- title: "Git Metrics"
url: "/repository/git-repository-size"
subnavigation:
- title: "Repository Size"
url: "/repository/git-repository-size"
- title: "GitHub Metrics"
url: "/repository/github-git-traffic"
subnavigation:
- title: "Git Traffic"
url: "/repository/github-git-traffic"
- title: "API Requests"
url: "/repository/github-api-requests"
45 changes: 45 additions & 0 deletions docs/_includes/repository_navigation.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
{% include environment.html %}
<section class="navigation">
{% if site.github.repository_nwo != nil %}
{% assign site.baseurl = site.github.repository_nwo %}
{% endif %}
{% assign activeItem = nil %}
{% assign activeSubitem = nil %}
{% for item in site.data.repository_navigation %}
{% if item.url == page.url %}
{% assign activeItem = item %}
{% endif %}
{% for subitem in item.subnavigation %}
{% if subitem.url == page.url %}
{% assign activeItem = item %}
{% assign activeSubitem = subitem %}
{% endif %}
{% endfor %}
{% endfor %}
<div class="back-link">&#8617; <a href="../repos-monitored" data-no-nwo-query>Back to instance statistics</a></div>
<ul class="navigation-primary">
{% for item in site.data.repository_navigation %}
<li>
{% if item == activeItem %}
<a href="{{ site.baseurl }}{{ item.url }}" class="active">{{ item.title }}</a>
{% else %}
<a href="{{ site.baseurl }}{{ item.url }}">{{ item.title }}</a>
{% endif %}
</li>
{% endfor %}
</ul>
{% if activeItem != nil and activeItem.subnavigation != nil %}
<ul class="navigation-secondary">
{% for subitem in activeItem.subnavigation %}
<li>
{% if subitem == activeSubitem %}
<a href="{{ site.baseurl }}{{ subitem.url }}" class="active">{{ subitem.title }}</a>
{% else %}
<a href="{{ site.baseurl }}{{ subitem.url }}">{{ subitem.title }}</a>
{% endif %}
</li>
{% endfor %}
</ul>
{% endif %}

</section>
61 changes: 61 additions & 0 deletions docs/_layouts/repository.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
{% include environment.html %}
<!doctype html>
<html lang="en-us">
<head>
<meta charset="UTF-8">
<title>Hubble Enterprise{% if page.title %}: {{ page.title }}{% endif %}</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="icon" sizes="256x256" href="{{ site.baseurl }}/assets/icons/favicon-256x256.png?version={{ site.github.build_revision }}">
<link rel="icon" sizes="192x192" href="{{ site.baseurl }}/assets/icons/favicon-192x192.png?version={{ site.github.build_revision }}">
<link rel="icon" sizes="128x128" href="{{ site.baseurl }}/assets/icons/favicon-128x128.png?version={{ site.github.build_revision }}">
<link rel="icon" sizes="96x96" href="{{ site.baseurl }}/assets/icons/favicon-96x96.png?version={{ site.github.build_revision }}">
<link rel="icon" sizes="64x64" href="{{ site.baseurl }}/assets/icons/favicon-64x64.png?version={{ site.github.build_revision }}">
<link rel="icon" sizes="48x48" href="{{ site.baseurl }}/assets/icons/favicon-48x48.png?version={{ site.github.build_revision }}">
<link rel="icon" sizes="32x32" href="{{ site.baseurl }}/assets/icons/favicon-32x32.png?version={{ site.github.build_revision }}">
<link rel="icon" sizes="16x16" href="{{ site.baseurl }}/assets/icons/favicon-16x16.png?version={{ site.github.build_revision }}">
<link rel="apple-touch-icon" sizes="57x57" href="{{ site.baseurl }}/assets/icons/apple-touch-icon-57x57.png?version={{ site.github.build_revision }}">
<link rel="apple-touch-icon" sizes="76x76" href="{{ site.baseurl }}/assets/icons/apple-touch-icon-76x76.png?version={{ site.github.build_revision }}">
<link rel="apple-touch-icon" sizes="120x120" href="{{ site.baseurl }}/assets/icons/apple-touch-icon-120x120.png?version={{ site.github.build_revision }}">
<link rel="apple-touch-icon" sizes="152x152" href="{{ site.baseurl }}/assets/icons/apple-touch-icon-152x152.png?version={{ site.github.build_revision }}">
<link rel="apple-touch-icon" sizes="167x167" href="{{ site.baseurl }}/assets/icons/apple-touch-icon-167x167.png?version={{ site.github.build_revision }}">
<link rel="apple-touch-icon" sizes="180x180" href="{{ site.baseurl }}/assets/icons/apple-touch-icon-180x180.png?version={{ site.github.build_revision }}">
<link rel="stylesheet" type="text/css" href="{{ site.baseurl }}/assets/css/normalize.css?version={{ site.github.build_revision }}" media="screen">
<link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Open+Sans:400,400i,600,600i" media="screen">
<link rel="stylesheet" type="text/css" href="https://use.fontawesome.com/releases/v5.0.6/css/all.css" media="screen">
<link rel="stylesheet" type="text/css" href="{{ site.baseurl }}/assets/css/stylesheet.css?version={{ site.github.build_revision }}" media="screen">
<link rel="stylesheet" type="text/css" href="{{ site.baseurl }}/assets/css/repository.css?version={{ site.github.build_revision }}" media="screen">
<script src="{{ site.baseurl }}/assets/js/vendor/jquery-3.2.1.min.js"></script>
<script src="{{ site.baseurl }}/assets/js/vendor/d3.v4.min.js"></script>
<script src="{{ site.baseurl }}/assets/js/vendor/moment-with-locales-2.21.0.min.js"></script>
<script src="{{ site.baseurl }}/assets/js/vendor/Chart-2.7.3.min.js"></script>
<script src="{{ site.baseurl }}/assets/js/vendor/spin-2.3.2.min.js"></script>
<script src="{{ site.baseurl }}/assets/js/repository.js?version={{ site.github.build_revision }}"></script>
<script src="{{ site.baseurl }}/assets/js/charts.js?version={{ site.github.build_revision }}"></script>
</head>
<body>
<section class="page-header">
<a href="https://github.com/Autodesk/hubble" class="fork-banner">
<img alt="Fork me on GitHub" src="{{ site.baseurl }}/assets/images/fork-banner.png">
</a>
<a href="{{ site.baseurl }}/" alt="Hubble Enterprise">
<img src="{{ site.baseurl }}/assets/images/logo-title.svg" class="project-logo" width="3rem" height="3rem">
<h1 class="project-name">
<script>
(function()
{
const queryString = new URLSearchParams(location.search);
const nwo = queryString.get('nwo');

document.write(`${nwo} Metrics`);
})();
</script>
</h1>
</a>
</section>
{% include repository_navigation.html %}
<section class="main-content">
{{ content }}
{% include footer.html %}
</section>
</body>
</html>
19 changes: 19 additions & 0 deletions docs/assets/css/repository.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
.page-header {
background-color: #275d9c;
}

.back-link {
position: absolute;
text-transform: uppercase;
font-size: 12px;
padding: 1.2rem 1.5rem;
text-decoration: none;
}

.back-link a {
text-decoration: none;
}

.back-link a:hover {
text-decoration: underline;
}
25 changes: 15 additions & 10 deletions docs/assets/js/_partials/chart-table.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
function createTable(table)
{
const url = $(table).data('url');
const noThead = $(table).data('no-thead');

let spinner = createSpinner(table);

Expand All @@ -13,14 +14,15 @@ function createTable(table)
if (data.length == 0)
return;

d3.select(table)
.append('thead')
.append('tr')
.selectAll('th')
.data(data.columns)
.enter()
.append('th')
.text(d => d);
if (noThead === undefined)
d3.select(table)
.append('thead')
.append('tr')
.selectAll('th')
.data(data.columns)
.enter()
.append('th')
.text(d => d);

let displayData = data;
const configSlice = readConfig($(table), 'slice');
Expand Down Expand Up @@ -61,9 +63,12 @@ function createTable(table)
case 'repository':
case 'resource':
case 'user':
const tableLinkHref = d3.select(table).attr('data-link-href');
const tableLinkTarget = d3.select(table).attr('data-link-target');

let a = cell.append('a').text(entry)
.attr('target', '_blank')
.attr('href', gheUrl() + '/' + entry)
.attr('target', tableLinkTarget || '_blank')
.attr('href', tableLinkHref ? tableLinkHref + entry : gheUrl() + '/' + entry)
.text(entry);

const tableID = d3.select(table).attr('id');
Expand Down
23 changes: 23 additions & 0 deletions docs/assets/js/repository.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
$(window).bind('load', function()
{
const queryString = new URLSearchParams(location.search);
const nwo = queryString.get('nwo');

// Add nwo query string to all navigation links
$('.navigation a').each(function()
{
const href = $(this).attr('href');
const noNwoQuery = $(this).attr('data-no-nwo-query')

if (noNwoQuery === undefined) {
$(this).attr('href', `${href}?nwo=${nwo}`);
}
});

// Replace all canvas data urls with repository specific urls
$('[data-url]').each(function()
{
const dataUrl = $(this).attr('data-url');
$(this).attr('data-url', dataUrl.replace('/repository', `/repository/${nwo}`));
});
});
2 changes: 2 additions & 0 deletions docs/demo-data/repository-monitored.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
repository
test-org/large-repo
6 changes: 6 additions & 0 deletions docs/demo-data/repository/test-org/large-repo/git-sizer.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
date commits commits [MiB] trees trees [MiB] tree entries blobs blobs [MiB] annotated tags refs biggest commit [KiB] most parents most tree entries biggest blob [MiB] largest history depth largest tag depth directories path depth path length files size of files [MiB] symlinks submodules
2019-01-01 800564 585.22 3732337 9638.486 253108785 1330280 60880.353 53 341 72.7 66 1895 13.496 145316 0 4434 13 134 63153 774.755 40 3
2019-01-02 803564 589.22 3752337 9838.486 272108785 1420280 61280.353 53 360 72.7 66 1895 13.496 145316 0 4434 13 134 63153 774.755 40 3
2019-01-03 806564 595.22 3772337 9938.486 285108785 1550280 62480.353 54 370 72.7 66 1895 13.496 145316 0 4434 13 134 63153 774.755 40 3
2019-01-04 809564 500.22 3792337 10038.486 290108785 1770280 63380.353 55 350 72.7 66 1895 13.496 145316 0 4434 13 134 63153 774.755 40 4
2019-01-05 811564 605.22 3832337 10638.486 303108785 1830280 64880.353 55 360 72.7 66 1895 13.496 145316 0 4434 13 134 63153 774.755 40 4
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
type count
"/repositories/:repository_id/issues/:issue_number/comments" 10620
"/repositories/:repository_id/issues/:issue_number" 9624
"/repositories/:repository_id/pulls/:pull_number" 9559
"/repositories/:repository_id/pulls" 1599
"/repositories/:repository_id" 291
"/repositories/:repository_id/contents/?*" 220
"/repositories/:repository_id/branches" 130
"/repositories/:repository_id/commits" 122
"/repositories/:repository_id/languages" 120
"/repositories/:repository_id/commits/*" 119
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
repository user cloning? requests download [GB]
test-org/large-repo someUser5322 true 7 37
test-org/large-repo someUser5337 true 44 36
test-org/large-repo someUser2762 true 40 34
test-org/large-repo someUser8267 true 27 28
test-org/large-repo someUser1992 true 35 26
test-org/large-repo someUser2706 true 110 23
test-org/large-repo someUser5302 true 27 22
test-org/large-repo someUser5357 true 80 21
test-org/large-repo someUser5317 true 139 18
test-org/large-repo someUser4337 true 76 16
test-org/large-repo someUser5137 true 74 11
test-org/large-repo someUser3589 true 10 10
test-org/large-repo someUser5134 true 33 9
test-org/large-repo someUser1528 true 63 9
test-org/large-repo someUser5737 true 43 7
test-org/large-repo someUser2064 true 6 6
test-org/large-repo someUser2736 true 40 6
test-org/large-repo someUser5550 true 7 6
test-org/large-repo someUser5347 true 129 6
test-org/large-repo someUser2434 true 20 6
test-org/large-repo someUser3526 true 21 6
test-org/large-repo someUser4112 true 62 6
test-org/large-repo someUser5307 false 29 5
test-org/large-repo someUser1322 true 68 5
test-org/large-repo someUser1972 true 33 5
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
date clones fetches clone traffic [GB] fetch traffic [GB] LFS traffic [GB]
2017-10-04 1722 2248 67 5 48
2017-10-03 1867 2123 57 8 40
2017-10-02 1605 2744 52 2 48
2017-10-01 608 359 18 2 8
2017-09-30 509 224 17 0 8.8
2017-09-29 1962 2087 65 2 45
2017-09-28 2352 2783 62 7 48
2017-09-26 2113 2656 73 4 44
2017-09-25 1840 2967 60 7 40
2017-09-24 798 676 27 9 8.6
2017-09-23 517 242 16 3 8
2017-09-22 1498 2414 56 3 44
2017-09-21 2035 2739 76 5 41
2017-09-20 1719 2829 87 9 46
2017-09-19 1864 2432 84 5 48
2017-09-18 1995 2742 89 6 46
2017-09-17 633 536 15 2 17
2017-09-16 526 263 13 1 8.2
3 changes: 3 additions & 0 deletions docs/demo-data/repository/test-org/large-repo/overview.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
key value
Repository Name test-org/large-repo
Repository Size 2340 MB
16 changes: 16 additions & 0 deletions docs/repos-monitored.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
layout: default
title: Monitored Repository
permalink: /repos-monitored
---

<div class="chart-placeholder">
<h3>Monitored Repositories</h3>
<table
data-url="{{ site.dataURL }}/repository-monitored.tsv"
data-type="table"
data-no-thead
data-link-href="./repository/?nwo="
data-link-target="_self"
></table>
</div>
Loading