Are arbitrary INSERT's supported?

[SamuelSarle]

Greetings,
Could you please help me understand the capabilities and limitations of Qlever a bit? I'd like to be able to insert data but this doesn't seem to be supported yet.

I do the following steps to reproduce the exception "exception": "Assertion `blocks.size() <= 1` failed. Should only be called for small relations. Please report this to the developers. In file \"/qlever/src/index/CompressedRelation.cpp \" at line 1513":

# create a directory
mkdir qlever-insert-test && cd qlever-insert-test

# pull latest qlever image
docker pull adfreiburg/qlever:latest

# start an interactive qlever container
docker run -it --rm -u $(id -u):$(id -g) -p 8888:8888 -v $(pwd):/data -w /data adfreiburg/qlever:latest

Now inside the container:

# add a default Qleverfile
qlever setup-config default

And either inside the container shell or on the host, use your favourite $EDITOR to modify the Qleverfile and update data.NAME and index.INPUT_FILES as follows:

[data]
NAME = "qlever-insert-test"

[index]
INPUT_FILES = *.nt

Create a file initial.nt that contains some data:

<http://example.org/subject/me> <http://example.org/predicate/name> "Me!"@en

Returning to the interactive shell inside the container:

qlever index
qlever start
qlever log

So the container shell is now following the server logs and I do this in another terminal window on the host machine:

# now we're able to query
curl localhost:8888/ --data-urlencode 'query=SELECT ?subject ?name WHERE { ?subject <http://example.org/predicate/name> ?name }'
# result =>
# {"head":{"vars":["subject","name"]},"results":{"bindings":[{"subject":{"type":"uri","value":"http://example.org/subject/me"},"name":{"type":"literal","value":"Me!","xml:lang":"en"}}]}}

# fetch the access token from Qleverfile
export ACCESS_TOKEN=$(grep '^ACCESS_TOKEN' Qleverfile | awk '{print $3}')

# insert data
curl localhost:8888/ --data-urlencode 'update=INSERT DATA { <http://example.org/subject/you> <http://example.org/predicate/name> "You!"@en }' --data-urlencode "access-token=$ACCESS_TOKEN"
# result =>
# Update successful

# query again
curl localhost:8888/ --data-urlencode 'query=SELECT ?subject ?name WHERE { ?subject <http://example.org/predicate/name> ?name }'
# result =>
# {
#    "exception": "Assertion `blocks.size() <= 1` failed. Should only be called for small relations. Please report this to the developers. In file \"/qlever/src/index/CompressedRelation.cpp \" at line 1513",
#    "query": "SELECT ?subject ?name WHERE { ?subject <http://example.org/predicate/name> ?name }",
#    "resultsize": 0,
#    "status": "ERROR",
#    "time": {
#        "computeResult": 2,
#        "total": 2
#    }
#}

The shell with qlever log open shows:

2025-01-01 14:10:05.656 - INFO: Request received via POST, content type "application/x-www-form-urlencoded"
2025-01-01 14:10:05.656 - INFO: Requested media type of result is "application/sparql-results+json"
2025-01-01 14:10:05.656 - INFO: Processing the following SPARQL query:
SELECT ?subject ?name WHERE { ?subject <http://example.org/predicate/name> ?name }
2025-01-01 14:10:05.658 - INFO: Query planning done in 1 ms
2025-01-01 14:10:05.658 - INFO: Result has size 1 x 2
2025-01-01 14:10:05.659 - INFO: Done processing query and sending result, total time was 3 ms
2025-01-01 14:12:39.209 - INFO:
2025-01-01 14:12:39.209 - INFO: Request received via POST, content type "application/x-www-form-urlencoded"
2025-01-01 14:12:39.210 - INFO: Processing the following SPARQL update:
INSERT DATA { <http://example.org/subject/you> <http://example.org/predicate/name> "You!"@en }
2025-01-01 14:12:39.212 - INFO: Query planning done in 2 ms
2025-01-01 14:12:39.212 - INFO: Done processing update, total time was 3 ms
2025-01-01 14:13:07.218 - INFO:
2025-01-01 14:13:07.219 - INFO: Request received via POST, content type "application/x-www-form-urlencoded"
2025-01-01 14:13:07.219 - INFO: Requested media type of result is "application/sparql-results+json"
2025-01-01 14:13:07.219 - INFO: Processing the following SPARQL query:
SELECT ?subject ?name WHERE { ?subject <http://example.org/predicate/name> ?name }
2025-01-01 14:13:07.220 - INFO: Query planning done in 1 ms
2025-01-01 14:13:07.221 - ERROR: Assertion `blocks.size() <= 1` failed. Should only be called for small relations. Please report this to the developers. In file "/qlever/src/index/CompressedRelation.cpp " at line 1513

Image:

docker image ls adfreiburg/qlever
#REPOSITORY          TAG       IMAGE ID       CREATED       SIZE
#adfreiburg/qlever   latest    30b1bbb026ac   2 weeks ago   641MB

Also environment info, although I don't think it's relevant here:

docker --version
#Docker version 27.4.0, build bde2b89

system_profiler SPSoftwareDataType SPHardwareDataType
#Software:
#    System Software Overview:
#      System Version: macOS 15.2 (24C101)
#      Kernel Version: Darwin 24.2.0

So the following two questions arise:

1. Am I missing something here or does Qlever not support such use yet where arbitrary data is inserted after the initial indexing? The above steps do work if the INSERT DATA contains only terms that are in the vocabulary so I assume that the vocab is the source of the error.
2. Even if I didn't get this error for the SELECT's subsequent to the INSERT, is my understanding correct that the inserts/updates are not persistent yet? The index and vocabulary files on disk have the original modified timestamp from when the qlever index was run and data disappears if I restart the server.

Relevant links:

1. Use DeltaTriplesManager to process update request #1608
2. Proof of concept for SPARQL UPDATE #916
3. https://github.com/ad-freiburg/qlever/wiki/QLever-support-for-SPARQL-1.1-Update
4. https://github.com/ad-freiburg/qlever/wiki/First-tests-with-SPARQL-1.1-Update
5. Update Index data #1610

[ktk]

This works for us:

# In case you want to get the access token from the server container (example from the local profile)
QLEVER_SERVER_ACCESS_TOKEN=$(docker compose --profile local exec -it server-local sh -c 'echo $QLEVER_SERVER_ACCESS_TOKEN')

# In case you have the query inlined
QUERY=$(cat <<EOF
INSERT DATA {
  <http://example.org/subject/you>   <http://example.org/predicate/name>  "You!"@en .
}
EOF
)

# In case you have the query in a file
QUERY=$(cat /path/to/query.rq)

# Execute the query
curl localhost:7001/ --data-urlencode "update=${QUERY}" --data-urlencode "access-token=${QLEVER_SERVER_ACCESS_TOKEN}"

[hannahbast]

@SamuelSarle Very sorry for the late answer.

@1: QLever does support the insertion of triples or quads with arbitrary literals and IRIs, and not only those that were already contained in the original input data (which would be very limiting). It seems you have hit a bug and we will try to fix it next week (this week is super busy). Please remind us in due time in case we forget. According to a preliminary investigation: the bug should only occur for small data or when using a literal or IRI that is very rare in the original input, and even then only in special cases.

@2: Updates are not persistent yet. It's on our agenda for SPARQL Update from now until the end of March. Regarding updates, we are currently working on https://www.w3.org/TR/sparql11-http-rdf-update (see #1668) and various bug fixes and efficiency improvements

[SamuelSarle]

Perfect, thank you for answering!