Skip to content
Back to search

sparql-llm

🦜✨ Chat system, MCP server, and reusable components to improve LLMs capabilities when generating SPARQL queries

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/sib-swiss--sparql-llm/badge.svg)](/listings/sib-swiss--sparql-llm)

HTML

<a href="/listings/sib-swiss--sparql-llm"><img src="/api/listings/sib-swiss--sparql-llm/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

> Requires Python >=3.10

pip install sparql-llm

Or with `uv`:

uv add sparql-llm

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md Β· README Β· 449 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. <div align="center">
  2. # ✨ SPARQL query generation with LLMs 🦜
  3. [![PyPI - Version](https://img.shields.io/pypi/v/sparql-llm.svg?logo=pypi&label=PyPI&logoColor=silver)](https://pypi.org/project/sparql-llm/)
  4. [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/sparql-llm.svg?logo=python&label=Python&logoColor=silver)](https://pypi.org/project/sparql-llm/)
  5. [![Tests](https://github.com/sib-swiss/sparql-llm/actions/workflows/test.yml/badge.svg)](https://github.com/sib-swiss/sparql-llm/actions/workflows/test.yml)
  6. </div>
  7. This project provides tools to enhance the capabilities of Large Language Models (LLMs) in generating [SPARQL](https://www.w3.org/TR/sparql11-overview/) queries for specific endpoints:
  8. - a complete **chat web service** available at **[expasy.org/chat](https://expasy.org/chat)**
  9. - a **MCP server** exposing tools at **[chat.expasy.org/mcp](https://chat.expasy.org/mcp)**
  10. - **reusable components** published as the **[`sparql-llm`](https://pypi.org/project/sparql-llm/)** pip package
  11. The system integrates Retrieval-Augmented Generation (RAG) and SPARQL query validation through endpoint schemas, to ensure more accurate and relevant query generation on large scale knowledge graphs.
  12. The components are designed to work either independently or as part of a full chat-based system that can be deployed for a set of SPARQL endpoints. It **requires endpoints to include metadata** such as [SPARQL query examples](https://github.com/sib-swiss/sparql-examples) and endpoint descriptions using the [Vocabulary of Interlinked Datasets (VoID)](https://www.w3.org/TR/void/), which can be automatically generated using the [void-generator](https://github.com/JervenBolleman/void-generator).
  13. ## πŸ’₯ News
  14. πŸ₯‡ We won the first place at the [Text2SPARQL Challenge](https://aksw.github.io/text2sparql.aksw.org/latest/results), co-located with ESWC 2026. Some highlights from our evaluation:
  15. - 🎯 up to 59% higher F1 score*
  16. - πŸš€ up to 27Γ— faster*
  17. - πŸ’Έ no more than $0.01 per question
  18. - 🌍 works across English, Spanish, and German
  19. *compared with the second-best system in the challenge
  20. ## 🌈 Features
  21. - **Metadata Extraction**: Functions to extract and load relevant metadata from SPARQL endpoints. These loaders are compatible with [LangChain](https://python.langchain.com) but are flexible enough to be used independently, providing metadata as JSON for custom vector store integration.
  22. - **SPARQL Query Validation**: A function to automatically parse and validate federated SPARQL queries against the VoID description of the target endpoints.
  23. - **MCP server** with tools to help LLM write SPARQL queries for a set of endpoints
  24. - **Deployable Chat System**: A reusable and containerized system for deploying an LLM-based chat service with a web UI, API, and vector database. This system helps users write SPARQL queries by leveraging endpoint metadata (WIP).
  25. - **Live Example**: Configuration for **[expasy.org/chat](https://expasy.org/chat)**, an LLM-powered chat system supporting SPARQL query generation for endpoints maintained by the [SIB](https://www.sib.swiss/).
  26. > [!TIP]
  27. >
  28. > You can quickly check if an endpoint contains the expected metadata at [sib-swiss.github.io/sparql-editor/check](https://sib-swiss.github.io/sparql-editor/check)
  29. ## πŸ”Œ MCP server
  30. The server exposes a [Model Context Protocol (MCP)](https://modelcontextprotocol.io/) endpoint to access [biodata resources](https://www.expasy.org/) at the [SIB](https://www.sib.swiss/), through their [SPARQL](https://www.w3.org/TR/sparql12-query/) endpoints, such as UniProt, Bgee, OMA, SwissLipids, Cellosaurus at **[chat.expasy.org/mcp](https://chat.expasy.org/mcp)**
  31. ### πŸ› οΈ Available tools
  32. - **πŸ“ Retrieve relevant documents** (query examples and classes schema) to help writing SPARQL queries to access SIB biodata resources
  33. - Arguments:
  34. - `question` (string): the user's question
  35. - `potential_classes` (list[string]): high level concepts and potential classes that could be found in the SPARQL endpoints
  36. - `steps` (list[string]): split the question in standalone smaller parts if relevant
  37. - **🏷️ Retrieve relevant classes schema** to help writing SPARQL queries to access SIB biodata resources
  38. - Arguments:
  39. - `classes` (list[string]): high level concepts and potential classes that could be found in the SPARQL endpoints
  40. - πŸ“‘ **Execute a SPARQL query** against a SPARQL endpoint
  41. - Arguments:
  42. - `query` (string): a valid SPARQL query string
  43. - `endpoint` (string): the SPARQL endpoint URL to execute the query against
  44. UniProt tools calling the [UniProt REST API](https://rest.uniprot.org) directly, defined in [`tools/uniprot.py`](src/sparql_llm/tools/uniprot.py):
  45. - **πŸ”Ž `search_proteins`**: search UniProtKB entries (accession, name, genes, organism)
  46. - **🧬 `get_protein`**: get a UniProtKB entry by accession (name, organism, function, sequence)
  47. UniProt tools showing interactive widgets to the user, for clients supporting [MCP Apps](https://modelcontextprotocol.io/specification/draft/extensions/apps). The widget gets the full data, the model only gets a text summary. Widgets are in [`widgets/`](widgets), each bundled with its dependencies in a single HTML file in `src/sparql_llm/widgets/` with `cd widgets && npm ci && npm run build` (done by the Dockerfile when building the image):
  48. - **`show_protein_features`**: [Nightingale](https://github.com/ebi-webcomponents/nightingale) tracks of domains, sites, PTMs, processing, topology, secondary structure and variants, zoomable along the sequence. Given `start` and `end` residues (e.g. 102 and 234, or only `start` for a mutation), the viewer opens zoomed on the region, highlighted on all tracks. Given `categories` (e.g. `["PTMs"]`), only them are shown when it opens
  49. - **`show_protein_structure`**: AlphaFold model colored by pLDDT, or a PDB entry, in Mol\* ([PDBe Molstar](https://github.com/molstar/pdbe-molstar)), next to the features list: clicking one highlights its residues. Residues can be highlighted from the start, e.g. a mutation, and the list filtered by `category` and `search`
  50. - **`search_proteins_table`**: results of a UniProtKB query (up to 1000 rows) in a sortable table with facets for reviewed status, annotation score and organism. The rows selected by the user are sent back to the model
  51. - **`show_protein_variants`**: disease and natural variants from the [EBI Proteins API](https://www.ebi.ac.uk/proteins/api/doc/) positioned on the sequence next to the domains, filterable by consequence, clinical significance, evidence and disease, with links to ClinVar, dbSNP and PubMed. Given `start` and `end` residues, it opens zoomed on the region like the features viewer, and the table lists the variants in view. The model can give the filters checked when it opens: `significance` (e.g. `["Pathogenic", "Likely pathogenic"]`), `consequence`, `evidence`, `disease` (matched on part of the name) and a `search` text
  52. - **`show_annotation_matrix`**: heatmap of the reviewed proteins of a gene list by GO terms, keywords or Reactome pathways, for a quick functional overview. Opens on the `annotation_type` given, with terms filtered by `search`
  53. - **`show_protein_network`**: graph connecting a protein list through 3 kinds of sourced relationships: directed regulations (activation, inhibition, or mixed when both are reported) from [OmniPath](https://omnipathdb.org) with their resources, PubMed references and modification sites, physical interactions from IntAct (via UniProtKB), and Reactome pathway membership. It shows the best supported paths from a source to a target protein (highest minimum evidence, then shortest), and the feedback edges going back upstream with the loop they close. Proteins not listed are added when they connect 2 listed ones. Each relationship links to its source records: OmniPath records and sites, PubMed references with the resources citing them, IntAct evidence (PSI-MITAB) and UniProtKB interactions, Reactome pathways, and the queries of all the data used. Drawn with [Cytoscape.js](https://js.cytoscape.org), in layers following the signal from the source
  54. Widget tools for other SIB resources, each in its own module:
  55. - **`show_gene_expression`** ([`tools/bgee.py`](src/sparql_llm/tools/bgee.py)): heatmap of the anatomical entities and cell types where genes are expressed, from the [Bgee API](https://www.bgee.org), colored by expression score, with organ systems, not expressed calls, data types and call quality, like the Bgee expression graph. Opens with the filters given: `organ_systems`, `data_type`, `gold_only`, `show_absent` and `search`
  56. - **`show_reaction`** ([`tools/rhea.py`](src/sparql_llm/tools/rhea.py)): [Rhea](https://www.rhea-db.org) reaction with its atom map drawn by Rhea (atoms of each reactant colored in the products, with their numbers, toggled like on the Rhea website), the structures of its participants, EC numbers, directions, cross-references and its reviewed enzymes in UniProtKB. Takes a Rhea ID or a Rhea search, e.g. `ec:2.5.1.18`
  57. - **`show_metanetx`** ([`tools/metanetx.py`](src/sparql_llm/tools/metanetx.py)): [MetaNetX](https://www.metanetx.org) chemical or reaction, from its SPARQL endpoint, with a graph of its identifiers in ChEBI, KEGG, BiGG, SEED, MetaCyc, Rhea, HMDB, SwissLipids... colored by how each one is reconciled (same structure, other protonation or stereo, obsolete). Chemicals show their structure, isomers and the reactions in the most models, reactions their participants and compartments, and the genome-scale metabolic models containing them with their enzymes
  58. - **`show_lipid`** ([`tools/swisslipids.py`](src/sparql_llm/tools/swisslipids.py)): [SwissLipids](https://www.swisslipids.org) lipid in the lipid hierarchy drawn as a graph (category > class > species > molecular, structural and isomeric subspecies), with its structure, chains, Rhea reactions and enzymes, own and inherited from its classes, and the tissues and cellular components it was found in
  59. - **`show_cell_line`** ([`tools/cellosaurus.py`](src/sparql_llm/tools/cellosaurus.py)): [Cellosaurus](https://www.cellosaurus.org) cell line with its identity, a warning when it is contaminated or misidentified, its parent, children and lines from the same individual, genome ancestry, mutations, HLA typing, and its STR profile compared allele by allele with the cell lines with the most similar profiles from [CLASTR](https://www.cellosaurus.org/str-search/), to check its authenticity
  60. - **`show_gene_family`** ([`tools/oma.py`](src/sparql_llm/tools/oma.py)): history of the gene family of a protein from its [OMA](https://omabrowser.org) Hierarchical Orthologous Group, parsed from its OrthoXML: the species tree with the duplications and losses of each clade and the genes of each species colored by paralog lineage, like the OMA iHam viewer, the counts of pairwise orthologs, and the gene neighbourhood of the protein and of its 1:1 orthologs, aligned and colored by gene family like the OMA local synteny viewer
  61. - **`show_ortholog_group`** ([`tools/orthodb.py`](src/sparql_llm/tools/orthodb.py)): [OrthoDB](https://www.orthodb.org) ortholog group of a protein at a taxonomic level, with its phyletic profile (species with the gene and copies per clade, down to each species), evolutionary descriptors, copy number histogram, functional annotations, and the groups of the broader levels and the groups it splits into, opened in place
  62. The buttons of the widgets send a prompt opening the other viewers, e.g. the MetaNetX identifiers of a Rhea participant, the atom map of a lipid reaction, or the protein viewers of an enzyme.
  63. ### ⚑️ Connect client to MCP server
  64. Follow the instructions of your client, and use the URL of the public server: **https://chat.expasy.org/mcp**
  65. For example, for GitHub Copilot in VSCode, to add a new MCP server through the VSCode UI:
  66. - [x] Open side panel chat (`ctrl+shift+i` or `cmd+shift+i`), and make sure the mode is set to `Agent` in the bottom right
  67. - [x] Open command palette (`ctrl+shift+p` or `cmd+shift+p`), and search for `MCP: Open User Configuration`, this will open a `mcp.json` file
  68. #### πŸ“‘ Use streamable HTTP server
  69. Connect to a running streamable HTTP MCP server, such as the publicly available [chat.expasy.org/mcp](https://chat.expasy.org/mcp).
  70. In your VSCode `mcp.json` you should have the following:
  71. ```sh
  72. {
  73. "servers": {
  74. "expasy-mcp-http": {
  75. "url": "https://chat.expasy.org/mcp/",
  76. "type": "http"
  77. }
  78. }
  79. }
  80. ```
  81. #### ⌨️ Use stdio transport
  82. ```sh
  83. uvx sparql-llm
  84. ```
  85. Optionally you can provide the path to a custom settings JSON file to configure the server (e.g. the list of endpoints that will be indexed and available through the server), see the [`Settings` class](https://github.com/sib-swiss/sparql-llm/blob/main/src/sparql_llm/config.py) for detailed available settings.
  86. Example settings file for your MCP server deployment:
  87. ```json
  88. {
  89. "app_org": "Your organization",
  90. "app_topics": "genes, proteins, lipids, chemical reactions, and metabolomics data",
  91. "endpoints" : [
  92. {
  93. "label": "UniProt",
  94. "endpoint_url": "https://sparql.uniprot.org/sparql/",
  95. "description": "UniProt is a comprehensive resource for protein sequence and annotation data."
  96. },
  97. {
  98. "label": "Bgee",
  99. "description": "Bgee is a database for retrieval and comparison of gene expression patterns across multiple animal species.",
  100. "endpoint_url": "https://www.bgee.org/sparql/",
  101. "homepage_url": "https://www.bgee.org/"
  102. }
  103. ]
  104. }
  105. ```
  106. Example `mcp.json` file to add and configure the MCP server in a client (e.g. VSCode):
  107. ```json
  108. {
  109. "servers": {
  110. "expasy-mcp": {
  111. "type": "stdio",
  112. "command": "uvx",
  113. "env": {
  114. "SETTINGS_FILEPATH": "/Users/you/sparql-mcp.json"
  115. },
  116. "args": [
  117. "sparql-llm"
  118. ]
  119. }
  120. }
  121. }
  122. ```
  123. > [!IMPORTANT]
  124. >
  125. > Click on `Start` just on top of `"openroute-mcp"` to start the connection to the MCP server.
  126. >
  127. > You can click the wrench and screwdriver button πŸ› οΈ (`Configure Tools...`) to enable/disable specific tools
  128. > [!NOTE]
  129. >
  130. > More details available in [the VSCode MCP official docs](https://code.visualstudio.com/docs/copilot/chat/mcp-servers).
  131. ## πŸ“¦οΈ Reusable components
  132. ### Installation
  133. > Requires Python >=3.10
  134. ```bash
  135. pip install sparql-llm
  136. ```
  137. Or with `uv`:
  138. ```sh
  139. uv add sparql-llm
  140. ```
  141. ### SPARQL query examples loader
  142. Load SPARQL query examples defined using the SHACL ontology from a SPARQL endpoint. See **[github.com/sib-swiss/sparql-examples](https://github.com/sib-swiss/sparql-examples)** for more details on how to define the examples.
  143. ```python
  144. from sparql_llm import SparqlExamplesLoader
  145. loader = SparqlExamplesLoader("https://sparql.uniprot.org/sparql/")
  146. docs = loader.load()
  147. print(len(docs))
  148. print(docs[0].metadata)
  149. ```
  150. You can provide the examples as a file if it is not integrated in the endpoint, e.g.:
  151. ```python
  152. loader = SparqlExamplesLoader("https://sparql.uniprot.org/sparql/", examples_file="uniprot_examples.ttl")
  153. ```
  154. > Refer to the [LangChain documentation](https://python.langchain.com/v0.2/docs/) to figure out how to best integrate documents loaders to your system.
  155. > [!NOTE]
  156. >
  157. > You can check the completeness of your examples against the endpoint schema using [this notebook](https://github.com/sib-swiss/sparql-llm/blob/main/notebooks/compare_queries_examples_to_void.ipynb).
  158. ### SPARQL endpoint schema loader
  159. Generate a human-readable schema using the ShEx format to describe all classes of a SPARQL endpoint based on the [VoID description](https://www.w3.org/TR/void/) present in the endpoint. Ideally the endpoint should also contain the ontology describing the classes, so the `rdfs:label` and `rdfs:comment` of the classes can be used to generate embeddings and improve semantic matching.
  160. > [!TIP]
  161. >
  162. > Checkout the **[void-generator](https://github.com/JervenBolleman/void-generator)** project to automatically generate VoID description for your endpoint.
  163. ```python
  164. from sparql_llm import SparqlVoidShapesLoader
  165. loader = SparqlVoidShapesLoader("https://sparql.uniprot.org/sparql/")
  166. docs = loader.load()
  167. print(len(docs))
  168. print(docs[0].metadata)
  169. ```
  170. You can provide the VoID description as a file if it is not integrated in the endpoint, e.g.:
  171. ```python
  172. loader = SparqlVoidShapesLoader("https://sparql.uniprot.org/sparql/", void_file="uniprot_void.ttl")
  173. ```
  174. > The generated shapes are well-suited for use with a LLM or a human, as they provide clear information about which predicates are available for a class, and the corresponding classes or datatypes those predicates point to. Each object property references a list of classes rather than another shape, making each shape self-contained and interpretable on its own, e.g. for a *Disease Annotation* in UniProt:
  175. >
  176. > ```turtle
  177. > up:Disease_Annotation {
  178. > a [ up:Disease_Annotation ] ;
  179. > up:sequence [ up:Chain_Annotation up:Modified_Sequence ] ;
  180. > rdfs:comment xsd:string ;
  181. > up:disease IRI
  182. > }
  183. > ```
  184. ### Generate complete ShEx shapes from VoID description
  185. You can also generate the complete ShEx shapes for a SPARQL endpoint with:
  186. ```python
  187. from sparql_llm import get_shex_from_void
  188. shex_str = get_shex_from_void("https://sparql.uniprot.org/sparql/")
  189. print(shex_str)
  190. ```
  191. ### Validate a SPARQL query based on VoID description
  192. This takes a SPARQL query and validates the predicates/types used are compliant with the VoID description present in the SPARQL endpoint the query is executed on.
  193. This function supports:
  194. * federated queries (VoID description will be automatically retrieved for each SERVICE call in the query),
  195. * path patterns (e.g. `orth:organism/obo:RO_0002162/up:scientificName`)
  196. This function requires that at least one type is defined for each endpoint, but it will be able to infer types of subjects that are connected to the subject for which the type is defined.
  197. It will return a list of issues described in natural language, with hints on how to fix them (by listing the available classes/predicates), which can be passed to an LLM as context to help it figuring out how to fix the query.
  198. ```python
  199. from sparql_llm import validate_sparql_with_void
  200. sparql_query = """PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
  201. PREFIX up: <http://purl.uniprot.org/core/>
  202. PREFIX taxon: <http://purl.uniprot.org/taxonomy/>
  203. PREFIX orth: <http://purl.org/net/orth#>
  204. PREFIX obo: <http://purl.obolibrary.org/obo/>
  205. PREFIX lscr: <http://purl.org/lscr#>
  206. PREFIX genex: <http://purl.org/genex#>
  207. PREFIX sio: <http://semanticscience.org/resource/>
  208. SELECT DISTINCT ?humanProtein ?orthologRatProtein ?orthologRatGene
  209. WHERE {
  210. ?humanProtein a orth:Protein ;
  211. lscr:xrefUniprot <http://purl.uniprot.org/uniprot/Q9Y2T1> .
  212. ?orthologRatProtein a orth:Protein ;
  213. sio:SIO_010078 ?orthologRatGene ;
  214. orth:organism/obo:RO_0002162/up:name 'Rattus norvegicus' .
  215. ?cluster a orth:OrthologsCluster .
  216. ?cluster orth:hasHomologousMember ?node1 .
  217. ?cluster orth:hasHomologousMember ?node2 .
  218. ?node1 orth:hasHomologousMember* ?humanProtein .
  219. ?node2 orth:hasHomologousMember* ?orthologRatProtein .
  220. FILTER(?node1 != ?node2)
  221. SERVICE <https://www.bgee.org/sparql/> {
  222. ?orthologRatGene a orth:Gene ;
  223. genex:expressedIn ?anatEntity ;
  224. orth:organism ?ratOrganism .
  225. ?anatEntity rdfs:label 'brain' .
  226. ?ratOrganism obo:RO_0002162 taxon:10116 .
  227. }
  228. }"""
  229. issues = validate_sparql_with_void(sparql_query, "https://sparql.omabrowser.org/sparql/")
  230. print("\n".join(issues))
  231. ```
  232. ## πŸš€ Complete chat system
  233. > [!WARNING]
  234. >
  235. > To deploy the complete chat system right now you will need to fork/clone this repository, change the configuration in `src/sparql-llm/config.py` and `compose.yml`, then deploy with docker/podman compose. It can easily be adapted to use any LLM served through an OpenAI-compatible API.
  236. >
  237. Requirements: Docker, nodejs (to build the frontend), and optionally [`uv`](https://docs.astral.sh/uv/getting-started/installation/) if you want to run scripts outside of docker.
  238. 1. Explore and change the system configuration in `src/sparql-llm/config.py`
  239. 2. Create a `.env` file at the root of the repository to provide secrets and API keys:
  240. ```sh
  241. CHAT_API_KEY=NOT_SO_SECRET_API_KEY_USED_BY_FRONTEND_TO_AVOID_SPAM_FROM_CRAWLERS
  242. LOGS_API_KEY=SECRET_PASSWORD_TO_EASILY_ACCESS_LOGS_THROUGH_THE_API
  243. OPENROUTER_API_KEY=sk-YYY
  244. OPENAI_API_KEY=sk-proj-YYY
  245. ```
  246. 3. Optionally, if you made changes to it, build the chat UI webpage:
  247. ```sh
  248. cd chat-with-context
  249. npm i
  250. npm run build:demo
  251. cd ..
  252. ```
  253. > You can change the UI around the chat in `chat-with-context/demo/index.html`
  254. 4. **Start** the vector database and web server locally for development, with code from the `src` folder mounted in the container and automatic API reload on changes to the code (the widgets are bundled when building the image, rebuild it with `--build` after changing them):
  255. ```bash
  256. docker compose up
  257. ```
  258. * Chat web UI available at http://localhost:8000
  259. * OpenAPI Swagger UI available at http://localhost:8000/docs
  260. * Vector database dashboard UI available at http://localhost:6333/dashboard
  261. **In production**, you will need to make some changes to the `compose.prod.yml` file to adapt it to your server/proxy setup:
  262. ```bash
  263. docker compose -f compose.prod.yml up
  264. ```
  265. Then run the indexing script manually from within the container to index the SPARQL endpoints (need to do it once):
  266. ```sh
  267. docker compose -f compose.prod.yml exec api uv run src/sparql_llm/indexing/index_resources.py
  268. ```
  269. > All data from the containers are stored persistently in the `data` folder (e.g. vectordb indexes and endpoints metadata)
  270. > Conversations are logged in `data/logs/requests.jsonl`, one JSON line per request (group lines on `sessionId` to rebuild multi-turn conversations): full messages, steps, generated SPARQL query, tokens, cost and runtime. Those logs can be explored with [chat-logs-viewer](https://github.com/sib-swiss/chat-logs-viewer). Change the path with the `REQUESTS_LOGS_FILEPATH` env variable.
  271. > [!NOTE]
  272. >
  273. > Query the chat API:
  274. >
  275. > ```sh
  276. > curl -X POST http://localhost:8000/chat -H "Content-Type: application/json" -d '{"messages": [{"role": "user", "content": "What is the HGNC symbol for the P68871 protein?"}], "model": "mistralai/mistral-small-latest", "stream": true}'
  277. > ```
  278. > [!WARNING]
  279. >
  280. > **Experimental entities indexing**: it can take a lot of time to generate embeddings for millions of entities. So we recommend to run the script to generate embeddings on a machine with GPU (does not need to be a powerful one, but at least with a GPU, checkout [fastembed GPU docs](https://qdrant.github.io/fastembed/examples/FastEmbed_GPU/) to install the GPU drivers and dependencies)
  281. >
  282. > ```sh
  283. > docker compose up vectordb -d
  284. > VECTORDB_URL=http://localhost:6334 nohup uv run --extra gpu src/sparql_llm/indexing/index_entities.py --gpu &
  285. > ```
  286. >
  287. >Then move the entities collection containing the embeddings in `data/qdrant/collections/entities` before starting the stack
  288. ### πŸ₯‡ Benchmarks
  289. There are a few benchmarks available for the system:
  290. - The `tests/benchmark.py` script will run a list of questions and compare their results to a reference SPARQL queries, with and without query validation, against a list of LLM providers. You will need to change the list of queries if you want to use it for different endpoints. You will need to start the stack in development mode to run it:
  291. ```sh
  292. uv run --env-file .env tests/benchmark.py
  293. ```
  294. > It takes time to run and will log the output and results in `data/benchmarks`
  295. - Follow [these instructions](tests/text2sparql/README.md) to run the `Text2SPARQL Benchmark`.
  296. - For biodata benchmark:
  297. ```sh
  298. docker compose up -d
  299. VECTORDB_URL=http://localhost:6334 uv run tests/benchmark_biodata.py
  300. ```
  301. ## πŸ§‘β€πŸ« Tutorial
  302. There is a step by step tutorial to show how a LLM-based chat system for generating SPARQL queries can be easily built here: https://sib-swiss.github.io/sparql-llm
  303. ## πŸ§‘β€πŸ’» Contributing
  304. Checkout the [`CONTRIBUTING.md`](https://github.com/sib-swiss/sparql-llm/blob/main/CONTRIBUTING.md) page.
  305. ## πŸͺΆ How to cite this work
  306. If you reuse any part of this work, please cite at least one of our articles below:
  307. - [SPARQL-LLM: Real-Time SPARQL Query Generation from Natural Language Questions](https://doi.org/10.1145/3847195) (full paper)
  308. ```bibtex
  309. @article{10.1145/3847195,
  310. author = {Smeros, Panayiotis and Emonet, Vincent and Wang, Ruijie and Sima, Ana-Claudia and Mendes de Farias, Tarcisio},
  311. title = {SPARQL-LLM: Real-Time SPARQL Query Generation from Natural Language Questions},
  312. year = {2026},
  313. publisher = {Association for Computing Machinery},
  314. address = {New York, NY, USA},
  315. issn = {1559-1131},
  316. url = {https://doi.org/10.1145/3847195},
  317. doi = {10.1145/3847195},
  318. journal = {ACM Trans. Web},
  319. month = sep,
  320. keywords = {Large Language Models, SPARQL, TEXT2SPARQL, Bioinformatics, Knowledge Graphs}
  321. }
  322. ```
  323. - [LLM-based SPARQL Query Generation from Natural Language over Federated Knowledge Graphs](https://ceur-ws.org/Vol-3953/355.pdf) (short paper)
  324. ```bibtex
  325. @conference{emonet2025llm,
  326. title={LLM-based SPARQL Query Generation from Natural Language over Federated Knowledge Graphs},
  327. author={Emonet, Vincent and Bolleman, Jerven and Duvaud, Severine and Mendes de Farias, Tarcisio and Sima, Ana Claudia},
  328. year = 2025,
  329. note={CEUR-WS.org, online \url{https://ceur-ws.org/Vol-3953/355.pdf}},
  330. booktitle = {ISWC 2024 Special Session on Harmonising Generative AI and Semantic Web Technologies, November 13, 2024, Baltimore, Maryland},
  331. volume = 3953,
  332. series = {CEUR Workshop Proceedings},
  333. }
  334. ```
  335. <!-- mcp-name: io.github.sib-swiss/sparql-llm -->

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.