Skip to content

Latest commit

 

History

History
445 lines (370 loc) · 58.9 KB

README.md

File metadata and controls

445 lines (370 loc) · 58.9 KB
Raw

Wikibase resources

A collection of resources, overviews, links and knowlegde related to Wikibase, collected and curated by KB, national library of the Netherlands.

This page was originally extracted from the slides of the lecture Introduction Wikibase, the basics, for employeees of KB, national library of the Netherlands on 7 September 2023. This (rather long) slidedeck is available on Wikimedia Commons and Zenodo.

This overview is heavily inspired by the Wikibase knowledge graphs, A collection of open source tools and resources related to Wikibase knowledge graphs by Renat Shigapov. I've used his centralized overview many times to improve my understanding of the Wikibase universe, and hope to extend it via the overview below, and help others in a similar way.

Motivation

I created this page for several reasons:

  • As a textual summary of my (rather exuberant, visual) presentation of September 7th.
  • From my own experience (especially if you are new to Wikibase) I know that it can be very time-consuming to discover and understand the different corners of the Wikibase world, and to find your way in this rather confusing forrest/jungle. I hope this centralized knowledge hub can help others make that journey a little easier/faster.
  • I have only been able to acquire most of my Wikibase knowledge thanks to the openness and generosity of the international Wikibase community and their willingness to make knowledge findable, visible and reusable for free. So I think it is no more than 'good reciprocal decency' to contribute to the community this overview of centralized and summarized knowledge.
  • As a place to record my "public personal memory of interesting Wikibase stuff".

Contributing to this page

This page is maintained by Olaf Janssen, Wikimedia coordinator of KB. See his Wikidata user page and expert page on kb.nl for contact details.

I plan to improve & expand the overview in the future. If you would like to contribute, please let me know.

Reuse and licensing

This overview can be reused freely and openly, it is available under the CC-BY 4.0 license, so attribution is required. Use something like

Wikibase resources, Olaf Janssen & KB national library of the Netherlands, https://github.com/KBNLwikimedia/Wikibase-resources

Latest updates

Latest update: Version 0.3, 6 December 2023 (added Wikibase projects for medieval manuscripts)

Contents

Table of contents generated with markdown-toc

Recap Wikidata

This section ia a summary of the lecture Introduction to Wikidata (Zenodo) (in Dutch)

  • Wikidata (d.d. 7 Sept 2023) contains structured descriptions of 107 million things, since October 2012

  • Wikidata items with geo location (June 2023) - Older/other maps - Interactive map

  • What are the principles of Wikidata? (see also the Wikidata introduction)

    • Structured descriptions of things, eg. Eiffel Tower
    • Central storage (vs. distributed in data silos)
    • Multilingual (200+ languages) - Description of the Eiffel Tower in English, Dutch, Portuguese and Japanese etc.
    • Linked data
    • Open & free
      • Free, no trackers, no ads, no usage fee.
      • No copyright or database rights, all data is available under the CC0 license.
      • Everyone can reuse data: query, share, copy , edit, download, sell, etc.
      • Anyone can contribute, edit, add, improve, delete, merge data, etc. --> Community
    • Community
      • International
      • Pm. 24K editors
      • Under the flag of the Wikimedia Foundation. Wikidata is a sister project of Wikipedia, Wikimedia Commons etc.
    • For humans and machines
      • Human readable, human writable --> Data available via GUIs in HTML
      • Machine readable, machine writable --> Data available via APIs in JSON , XML/RDF, CSV etc.

  • Wikidata is a secondary, general purpose, public knowledge base for the world.

    • Secondary
      • Non-original data about
      • Notable things (more info)
      • Verifiable by reliable public sources (more info)
    • General purpose
      • Wide scope of topics/classes
      • Items contain relatively basic data (limited set of properties), Wikidata is not aimed at superspecialistic/deep data
    • Public
      • Public data
      • Without copyright issues
      • Without privacy issues (more info)

Institutional use cases incompatible with Wikidata

  • Institutional use cases for which Wikidata is not suitable
    • Publish domain-specific / specialist / 'esoteric' LOD
    • Publish very large LOD sets (e.g. catalogues, thesauri)
    • Use of very specific/complex/deep/layered data models
    • Control over who can add/change data
    • Collaboration with selected partners in a closed, controlled environment
    • Recording non-public data
    • Own control over hosting / IT infrastructure

What is Wikibase?

  1. The open-source, free software that powers Wikidata - "Wikibase is essentially a blank copy of Wikidata into which you can put your own structured data" (source),
  2. Allowing you to build & manage your own LOD knowledge base,
  3. Without the disadvantages of Wikidata,
    • You can create your own data models, as domain-specific/specialist/esoteric as you want/need.
    • Large data sets are no problem
    • Custom rights management (control over who is allowed to contribute)
    • Wikibase instances can be non-public
    • You can host your own instance
  4. With all the benefits of Wikidata
    • Focused on collaboration and connection (including Wikidata community)
    • For people and machines
    • User-friendly GUI for structured data
    • Native multilingualism support
    • Version history and control, rollbacks
    • Clear ontology: Items, Properties, Statements etc.
    • Output in various data formats (including JSON, RDF/XML, N3)
    • Search via SPARQL
    • Well suported and documented MediaWiki API
    • Support for tools (including OpenRefine)
    • Documentation for Wikidata is in general applicable to Wikibase as well

Wikibase courses and tutorials

See also Awesome Wikibase tutorials collected by Renat Shigapov.

Examples of institutions/projects using Wikibase

Wikibase projects in libraries

Wikibase is being evaluated by libraries as a tool to help them store and manage their structured data, as well as connect to the world of linked open data.

Wikibase projects for other GLAMs

  • GLAM network Luxembourg (more info)
  • Europeana EAGLE network - aims to build a multi-lingual online collection of millions of digitised items from European museums, libraries, archives and multi-media collections, which deal with the surviving inscriptions of the Greek-Roman world. The EAGLE Wikibase is designed to give a tool to anyone interested in bridging this gap and contributing translations of inscriptions.
  • Fotomuseum Antwerpen (more info)

Wikibase projects for medieval manuscripts

This section is extracted from the presentation Introduction to Wikibase for medieval manuscripts dd 6 December 2023. See the sections about Digital Scriptorium and Biblissima.

Digital Scriptorium

Biblissima

Wikibase components & architecture

Diefenbach et al. (2021), Wikibase as an Infrastructure for Knowledge Graphs: the EU Knowledge Graph, see "3.1 Wikibase infrastructure"

Wikibase data model

Using alternative vocabularies in Wikibase

  • As explained above Wikibase has its own unique data model, which has its limitations. To what extent can other vocabularies (such as RDA and Schema.org) be included into a Wikibase?
  • Literature explaining the limitations of the Wikibase model:
  • Analysis of the problem by Marieke Moolenaar (KB, August 2023)
    Wikibase uses a derivative of Blazegraph as its linked data storage. Let's call this the Wikibase-Blazegraph-DB. Blazegraph is a so-called graph DB or triple store, so you should be able to store RDF triples in it, thus also RDA/RDF triples and Schema.org triples. Currently, default Wikibase instances are set up in such a way that only Wikibase Q-P-Q triples can be included in the Wikibase-Blazegraph-DB via an update process from the MediaWiki-MySQL database. Schema.org or RDA/RDF triples can never enter the Wikibase-Blazegraph-DB via that update process. So if you want to get triples with other vocabulary into the KB-Wikibase-Blazegraph-DB, you will have to put them in there via another way than via the Wikibase-MediaWiki-GUI and the MediaWiki-MySQL database upate process.
  • Workaround by National library of Greece (NLG) - Implementing RDA in Wikibase, C. Bratsas and L. Ioannidis of Open Knowledge Greece
  • Summary of the NLG approach written by Marieke Moolenaar (KB, August 2023):
    The Greek Wikibase is only used as a graphical interface (front-end) for entering thesaurus data by library staff. They do not use their Wikibase to publish linked data, but only for data entry purposes. The reason they do not use Wikibase for publishing linked data is that in Wikibase you can only use the proprietary Wikibase metadata model, thus external vocabularies (such as RDA/RDF) cannot be used in Wikibase. To be able to enter new thesaurus triples via Wikibase screens/GUI, the NLG has made a translation/mapping from RDA/RDF to Wikibase entities (Ps and Qs). They periodically copy all created thesaurus triples from the Wikibase graph to another triple store (Triply). To do this, they translate their Wikibase triples back to RDA/RDF triples. In Triply they store and publish the real RDA/RDF vocabulary triples, which are not (or cannot be) present in Wikibase.
  • How to load up schema.org data dumps into Blazegraph by Dan Brickley, August 2016

Wikibase hosting

Requesting data from a Wikibase

1) HTML content in web browser

Theun de Vries in KB sandbox Wikibase GUI, 4 equivalent URLs

2) Non-HTML content in web browser

3) MediaWiki Action API

4) SPARQL

Cool Wikibase SPARQL queries

Adding data to a Wikibase

Useful links:

1) Add a new item via the GUI

Using KB's sandbox WB (login required) we can create a NewItem, resulting into an item about the Dutch poet H.H. ter Balkt

2) OpenRefine (in bulk)

3) QuickStatements (in bulk)

4) Advanced data import tools

Wikibase community

1) Wikibase Community User Group (WBCUG)

Its mission is to cultivate Wikibase's development and to encourage like-minded developers and data analysts not only to improve Wikibase's existing tools but also to create new ones.

About the WBCUG - WBCUG history - WBCUG members - WBCUG monthly online meetings - The Wikibase Live sessions - WBCUG Mailing list (archives) - WBCUG Telegram (or here)

2) Wikibase Stakeholder Group (WBSG)

Commissions production and maintenance of open source extensions to Wikibase, and documentation for institutions that want to operate and maintain a fully-fledged instance of Wikibase. The group will focus on extensions to Wikibase instead of contributing to Wikibase core.

About the WBSG - WBSG members - WBSG meeting calender - WBSG monthly online meeting minutes - WBSG Loomio - WBSG Mastodon - WBSG Twitter

3) Wikibase Knowlegde Group Netherlands (WBGNL)

The aim of this group is to bundle and exchange knowledge & experiences about the use of Wikibase, to learn from each other, and to keep each other informed about the (international) developments and opportunities surrounding Wikibase. Membership is open to everyone in the Netherlands who already works with Wikibase, wants to work with, or is otherwise interested in this software. Mainly for - but certainly not limited to - professionals from Dutch heritage and knowledge institutions, and related organizations and companies.

About the WBGNL - WBGNL meetings (also available here) - WBGNL mailing list - WBGNL Loomio

4) Wikibase community workshops, trainings, meetups etc. (archive)

Staying updated

  • Via the meetings, minutes, presentations, mailing lists, socials etc. of the WBCUG, WBSG and WBGNL. See above.
  • Wikibase.cloud: Project updates - Mailing list (archives) - Telegram
  • Specifically for libraries:
    • Wikibase Working Hours, community discussion of Wikidata & Wikibase with the goal of understanding how the library can contribute to and leverage these as a platform for publishing, linking, and enriching library linked data.
    • IFLA Wikidata Working Group. This working group will explore and advocate for the use of and contribution to Wikidata by library and information professionals, the integration of Wikidata and Wikibase with library systems, and alignment of the Wikidata ontology with library metadata formats such as BIBFRAME, RDA, and MARC.
  • More:

Finding help

Active help (by humans)

Passive help (documentation, manuals, HOWTOs, tutorials etc.)

More help