Comunica Bus SPARQL Parse

A bus for parsing an input query into SPARQL algebra.

This module is part of the Comunica framework, and should only be used by developers that want to build their own query engine.

Click here if you just want to query with Comunica.

Install

$ yarn add @comunica/bus-sparql-parse

Bus usage

• Context: "https://linkedsoftwaredependencies.org/bundles/npm/@comunica/bus-sparql-parse/^1.0.0/components/context.jsonld"
• Bus name: cbsp:Bus/SparqlParse

Creating actors on this bus

Actors extending ActorSparqlParse are automatically subscribed to this bus.

Changelog

All notable changes to this project will be documented in this file.

v1.22.0 - 2021-08-30

Added

• Add support for SPARQL extension functions
• Allow custom fetch function to be provided via context
• Support AbortControllers in native http actor
• Report FILTER errors as warnings in the logger, Closes #767
• Add SPARQL endpoint-based hypermedia update actor
• Allow resource creation via HTTP PUT:
  • Add RDF update hypermedia actor for LDP PUT destinations
  • Configure RDF update hypermedia actor for LDP PUT destinations
  • Allow RDF dereference actors to return on non-200 responses
  • Add metadata extractor for HTTP Accept-Put header
  • Add metadata extractor for HTTP Allow header

Changed

• Refactor CLI arguments handling using yargs
• Optimize node-fetch HTTP actor and set as default
• Include response body in dereference errors
• Bump fetch-sparql-endpoint with improved error reporting for SPARQL endpoints
• Update dependency htmlparser2 to v7 (#851)
• Migrate to @rdfjs/types
• Bump to sparqlalgebrajs 3
• Notify downstream clients on worker timeout
• Bump fetch-sparql-endpoint with AbortControllers
• Make HTTP accept headers deterministic
• Add URL-based heuristic to determine SPARQL endpoints to top-level actor

Fixed

• Update to AsyncIterator 3.2.0 to fix memory issue
• Fix http proxy not working with immutable node-fetch response
• Fix invalid hash implementation for hash-join
• Fix node-fetch actor not supporting body.cancel
• Do not reset sparql.js blank-node counter (Closes #836)
• Fix SPARQL-based update actors not being able to handle quads

v1.21.3 - 2021-06-18

Fixed

• Fix POST requests with body not working on browsers

v1.21.2 - 2021-06-14

Changed

• Interact with SPARQL endpoints via HTTP POST, Closes #814
• Update dependency @types/node to v14
• Update jsonld-streaming-parser to 2.3.2

v1.21.1 - 2021-04-27

Republish due to npm publishing failure of 1.21.0.

v1.21.0 - 2021-04-27

Added

• Enable hypermedia-driven updates
  • Pass headers to metadata extraction actors
  • Add metadata extractor for detecting SPARQL Update Patch servers
  • Configure hypermedia-based update actors
  • Add actor for handling servers supporting SPARQL Update patches
  • Add update actor for delegating to hypermedia bus
  • Add bus for hypermedia-based update actors
• Support HTTP requests with bodies
• Allow context modification in optimize-query-operation bus

Changed

• Move IQuadDestination to separate file
• Allow multiple graphs to be created/deleted in IQuadDestination
• Use --to option instead of -d to mark destination on CLI
• Allow any parser options to be passed to actor-rdf-parse-jsonld

Fixed

• Emit non-zero exit code on CLI errors, Closes #805
• Remove unnecessary dependencies on utils-datasource to mute Yarn warnings
• Fix types dependencies not being actual dependencies
• Fix unable to federate over multiple GraphDB instances

v1.20.0 - 2021-03-30

Added

• Add support for SPARQL Update queries
  • Add update actors to all init engines
  • Support update queries in SPARQL endpoint
  • Support update results in simple serializer
  • Add readOnly context entry that makes update actors fail
  • Allow single source in context to be inherited as destination
  • Add add, copy, and move update operation actors
  • Add create update operation
  • Add clear and drop update operations
  • Configure load update actor
  • Add load update query operation actor
  • Add fallback rdf-dereference actor
  • Run SPARQL Update test suite
  • Enable passing destination via the CLI
  • Configure INSERT and DELETE actors
  • Add composite update operation actor
  • Add INSERT/DELETE operation actor
  • Add RDF update actor that handles RDF/JS stores
  • Add bus for handling quad updates
  • Add update query operation output type
• Enable worker threads in SPARQL endpoints
• Delegate link queue creation to the new link queue bus

Changed

• Move public query engine interfaces into @comunica/types
• Move all context keys to '@comunica/context-entries'
• Keep track of parent link when pushing into link queue
• Save original query in context

Fixed

• Enable blank node correlation across results, Closes #795
• Fix FROM/FROM NAMED not always restricting views
• Bump asyncjoin with close on undefined fix, Closes #791
• Fix process hanging if a source parsing action rejects, Closes #786
• Fix hypermedia quad pattern resolver not skipping links to itself

v1.19.2 - 2021-02-02

Changed

• Remove disabling of stacktraces when in production

Fixed

• Fix unhandled promise rejection on server errors

v1.19.1 - 2021-01-22

Fixed

• Fix logger not always producing output, Closes #774

v1.19.0 - 2021-01-15

Changed

• Update to Components.js 4
• Update dependency htmlparser2 to v6

v1.18.1 - 2020-12-01

Fixed

• Fix HTTP service not handling conneg correctly with wildcards
• Remove unsafe usages of process
• Fix minor (non-breaking) issues with configs

v1.18.0 - 2020-11-02

Fixed

• Fix eager accept header boundary bug, Closes #534
• Fix incorrect return type for bindings(), Closes #742
• Add missing query result interface exports

Added

• Add Microdata to RDF parser
• Add actor whitelist to LoggerPretty

Changed

• Use fixed hashing functions
• Update sparqlee to achieve smaller bundle sizes
• Bump sparqlee with improved XPath and Unicode support
• Allow quad streams on followed links to be transformed
• Allow hypermedia links to define custom context entries
• Allow JSON-LD strict mode to be enabled via context
• Allow custom JSON-LD document loader to be passed via context

v1.17.0 - 2020-09-25

Added

• Include request headers in logs, Closes #703
• Include user agent in node-fetch HTTP actor

Changed

• Update to asynciterator 3.0.3 to fix hanging queries
• Make metadata retrieval more robust, and fix errors where metadata is emitted before end event
• Make logger data argument lazy
• Tweak priorities to let endpoint default to application/json, Closes #662
• Increase default Turtle priority to 0.6, higher than RDF/XML
• Update to @types/rdf-js 4
• Migrate to rdf-data-factory

Fixed

• Update to JSON-LD context parser with schema.org fix
• Fix redirect not including query parameter

v1.16.2 - 2020-08-24

Fixed

• Fix native HTTP actor broken in browser

v1.16.1 - 2020-08-24

Fixed

• Fix incorrect JSON-LD error code on target script error

v1.16.0 - 2020-08-24

Added

• Implement missing property path cases
• Add HTTP basic authentication support, Closes #613
• Add TSV SPARQL results serializer, #220
• Add CSV SPARQL results serializer, #220

Fixed

• Update sparql parser to fix all syntax spec test failures, #287
• Fix network request without headers, Closes #700 (#701)
• Correctly handle aggregates without group
• Fix subquery aggregates halting the process
• Fix variable graph also matching with default graph
• Fix hypermedia actor failing on * property path queries
• Fix property path queries failing with named graphs
• Fix mainModulePath overriding not being possible in dynamic exec
• Fix process hanging after HTTP error
• Fix HTTP error events being emitted twice
• Fix targeted HTML script tags not throwing error if of unknown type
• Fix missing newline after --listformats

Changed

• Configure nestedloop join actor for handling streams with undefs
• Annotate bindingsStreams with canContainUndefs flag

v1.15.0 - 2020-08-13

Added

• Add convenience methods for getting query results as arrays
• Add option to include credentials in browsers
• Enable min/max over non-numerical literal, Closes #673

Changed

• Improve command signature of comunica-sparql-http
• Remove lodash dependencies, Closes #402

Fixed

• Fix error when parsing webpage with JSON script tag, Closes #707
• Update to asynciterator 3.0.1 to undo webpack hack
• Add missing newline after each pretty log line
• Fix incorrect hash function in symmetric hash join, Closes #608
• Bump to sparqlee 1.4.2 with strtswith and endswith fix

v1.14.0 - 2020-07-24

Added

• Enable cardinality estimates in RDFJS sources
• Redirect default HTTP index to /sparql, Closes #663
• Simplify how RDFJS sources can be passed

Changed

• Migrate to asynciterator to 3 to improve performance in browsers
• Update dependency lru-cache to v6
• Move relevant @types to dependencies, Closes #294
• Replace json-stable-stringify dependency with canonicalize, Closes #590

Fixed

• Fix floating promises
• Fix test failures on Node 10
• Use latest fetch API with cross-fetch to fix fetch-related compatibility issues
• Fix orderby not taking numerical types into account
• Fix invalid RDFJS actor IRI
• Fix newEngine in SPARQL init file exporting browser build, Closes #648
• Fix orderby not properly handing multiple comparators, Closes #598

v1.13.1 - 2020-06-11

Fixed

• Fix crash when using CONSTRUCT queries
• Fix unasigned variables in projection result

v1.13.0 - 2020-06-03

Fixed

• Modify JSON-LD document loader to consider redirects
• Fix table serializer producing inconsistent column orders, Closes #643
• Update sparqlalgebra to fix invalid COUNT queries to endpoints

Changed

• Enable stricter TypeScript compilation, Closes #628

v1.12.1 - 2020-04-27

Fixed

• Fix EXISTS in BIND failing, Closes #650
• Fix project operator removing scoped blank nodes, solid/query-ldflex#64

v1.12.0 - 2020-04-03

Changed

• Update to JSON-LD parser to 1.1

Added

• Handle JSON-LD via JSON extension types as well
• Pass HTTP headers to HTML script parsers
• Allow HTML scripts to be targeted by id via fragments
• Throw error on application/json without valid JSON-LD link header

Fixed

• Fix HTML parser not forwarding errors within listeners

v1.11.1 - 2020-04-02

Added

• Add Runner.collectActors method

Fixed

• Fix initialBindings not applied on full query

Changed

• Output number of HTTP requests in stats serializer
• Remove unneeded action argument in Setup.instantiateComponent

v1.11.0 - 2020-03-30

Added

• Add more efficient optional actor
• Expose SPARQL service description in HTTP tool, Closes #219

Changed

• Optimize default graph handling in QPF sources
• Scope blank nodes to each federated source

Fixed

• Fix application aborting on some optional queries
• Use xsd:dateTime format (ISO 8601) for NOW
• Fix several issues with SPARQL endpoints:
  • Fix crash when slicing construct queries
  • Fix only SELECT queries being handleable by SPARQL endpoints
  • Fix SPARQL endpoint detection failing when SD uses blank node subjects #619
  • Fix SPARQL endpoint detection failing for relative IRIs #619
  • Consider sources ending with '/sparql' as SPARQL endpoints, Closes #535

v1.10.0 - 2019-12-12

Added

• Add actors for joining more than two streams pair-wise
• Allow quad-pattern-level contexts to be defined

Changed

• Manually load sources for federated metadata
• Clarify dereferencing error message.

Fixed

• Fix unresolved promise rejections in SPARQL tree serialization
• Swap node-web-streams package with web-streams-node
• Remove unused jsonld dependency

v1.9.4 - 2019-10-21

Changed

• Bump to Components.js 3.3.0

v1.9.3 - 2019-10-16

Added

• Add lenient mode for HTTP and parsing errors
• Store currently processing operation in context
• Log requesting URLs in node-fetch HTTP actor

Changed

• Tweak priorities of default RDF media types
• Make hypermedia link following lazy
• Allow headers and method to be passed to RDF dereferencers
• Update to Sparqlalgebra.js 2.x.x

Fixed

• Add missing stream-to-string to actor-rdf-parse-jsonld

v1.9.2 - 2019-09-27

Added

• Plug HTTP bus into JSON-LD parser, Closes #478
• Detect JSON-LD contexts in link headers, Closes #486
• Throw error when multiple JSON-LD link headers to contexts were detected

Changed

• Emit missing metadata as null
• Remove deprecated util.inspect calls, Closes #488

Fixed

• Fix QPF sources never going beyond page 2
• Reimplement MINUS operator to handle variables according to spec
• Ensure unique blank nodes across different bindings
• Fix sequence path operator actor missing a join mediator
• Filter quads from quad patterns with shared variables
• Fix SPARQL BASE not propagating through operators
• Fix actor-init-sparql bin being broken, Closes #493
• Fix various small Components.js naming issues
• Update sparqlee to 1.2.0 to fix precision errors in basic arithmetics

v1.9.1 - 2019-07-30

Fixed

• Fix file source not emitting metadata, resulting in slow queries
• Fix crash due to missing version bump in rdf-terms

v1.9.0 - 2019-07-29

Added

• Add HTML script tag parser
• Plug RDFa parse into HTML parsing bus, Closes #138
• Add RDF RDFa parser for XML documents
• Add HTTP proxy actor, Closes #465
• Allow actors to be registered before other actors in buses
• Add abstract GraphQL-LD-based hypermedia extractor
• Add query-based metadata extractors, Closes #115
• Add BusIndexed that indexes actors by type

Fixed

• Fix broken docker images, Closes #469
• Fix hashing breaking with different data factories, #438
• Fix GraphQL-LD query results not singularizing
• Fix RDF dereferencing not taking into account relative response URLs

Changed

• Add RDFJS source actor to default SPARQL config
• Bump sparqlee to 1.1.0, Closes #477
• Allow auto-sources to be defined as strings instead of objects
• Add hypermedia-based SPARQL resolver using service descriptions
• Log identified source
• Improve error message on RDF dereference HTTP errors
• Allow combine union mediator to have an empty bus
• Pass headers to RDF parse actors
• Append '/' in accept header when shortened, Closes #471
• Deprecate triple predicate metadata identifier
• Add config option to allow metadata to also be emitted as data
• Add fallback metadata actor that sees everything as both meta and data
• Make primary topic actor fallback to 'all data is also metadata'
• Remove heuristical source identification
• Emit dataset id in Hydra controls extractor
• Rewrite quad pattern hypermedia resolver to be less restrictive
• Add hypermedia resolver for unknown sources
• Refactor QPF-specific hypermedia resolver, #432
• Remove lodash.map from mediators
• Improve performance of applyInitialBindings
• Index the query operation bus
• Remove unneeded string trimming in SPARQL serialization

v1.8.0 - 2019-06-13

Changed

• Update sparqlee to 1.0.0
• Bump to new GraphQL-LD version

v1.7.4 - 2019-05-29

Fixed

• Fix RDF source identifier joining content types incorrectly
• Allow JSON-LD context to appear out-of-order

v1.7.3 - 2019-05-09

Fixed

• Fix bnodes in CONSTRUCT not working, Closes #449
• Fix CONSTRUCT on empty WHERE clause producing no 1 result, Closes #448

v1.7.2 - 2019-05-03

Fixed

• Remove console.log in HTTP parse RDF dereference actor

v1.7.1 - 2019-05-03

Fixed

• Restrict accept headers to 128 characters in browsers to fix CORS issues
• Fix native HTTP actor not forwarding request errors, Closes #381

Changed

• Rescale default media type priorities
• Remove unneeded spaces and zeros in accept headers
• Add CORS headers for SPARQL interface

v1.7.0 - 2019-04-11

Added

• Allow expressions in EXISTS
• Add support for SPARQL aggragates
• Allow baseIRI to be defined for queries

Fixed

• Fix local file querying not working

v1.6.6 - 2019-04-03

Changed

• Remove jsonld.js in favor of streaming JSON-LD processing, Closes #384
• Use real-world endpoint examples in query help message
• Return error code when config compilation failed, Closes #394
• Make SPARQL endpoint identifier use GET instead of HEAD
• Lower default tree serialization priority

Fixed

• Fix bug in orderby due to multiple transforms on same stream, Closes #434
• Fix URL hashes not being stripped when doing HTTP requests, Closes #277
• Fix invalid default REDUCED cache size, Closes #376
• Fix HEAD requests on HTTP service not working, Closes #417
• Fix patterns with conflicting variables being created in federation

v1.6.5 - 2019-03-15

Added

• Emit bound pattern metadata in BGP subcontexts

Fixed

• Fix invalid expression actor test functions

v1.6.4 - 2019-03-05

Fixed

• Fix HTTP service imports

v1.6.3 - 2019-02-27

Changed

• Fix performance issue due to many Error instantiations
• Disable empty sources check in federated actor

v1.6.2 - 2019-02-26

Added

• Add option to invalidate cache before each query exec

Changed

• Improve help message of HTTP tool

v1.6.1 - 2019-02-25

Fixed

• Fix http tool not working with absolute config paths

v1.6.0 - 2019-02-22

Added

• Expose invalidateHttpCache method on SPARQL init
• Support source identification for single sources
• Pass parent metadata to BGP and pattern actions

Changed

• Update sparqlee to version 0.1.0, improves FILTER expressivity
• Use LRU cache in file resolver
• Abstract SPARQL HTTP service

Fixed

• Fix config compilation issues in file init actor
• Fix parser/serializer priority values not being applied
• Fix not all media types being determined in RDF dereference, Closes #318
• Fix invalid unscoped JSON-LD objects

v1.5.4 - 2019-01-31

Fixed

• Fix newEngineDynamic failing to produce results after multiple calls
• Fix http tool not working with absolute config paths

Changed

• Update jsonld to 1.5.0

v1.5.3 - 2019-01-24

Fixed

• Fix GraphQL-LD queries not producing results anymore
• Fix schema and skos prefix

v1.5.2 - 2019-01-17

Fixed

• Fix extend operator still depending on old sparqlee version

v1.5.1 - 2019-01-17

Fixed

• Fix tslib dependency issue in sparqlee, causing a crash at startup
• Fix JSON-LD serializer producing invalid JSON
• Check HTTP stream type when identifying source
• Remove unneeded filter peer dependency from orderby actor
• Remove unneeded @types/bluebird dependency, Closes #360

v1.5.0 - 2019-01-02

Added

• Add Filter operator (based on Sparqlee)
• Add Extend operator (based on Sparqlee)
• Update LeftJoin NestedLoop actor to use Sparqlee
• Add BGP join optimizer, Closes #90
• Add query operation optimize bus

v1.4.6 - 2018-12-11

Fixed

• Fix missing dependencies between new hypermedia actors

v1.4.5 - 2018-12-11

Fixed

• Fix blank nodes in data being seen as variables
• Fix queries with a variable graph only matching the default graph
• Fix missing base IRI in JSON-LD parser, Closes #342

Changed

• Split resolve quad pattern actor into two separate modules

v1.4.4 - 2018-11-13

Fixed

• Update to Components.js 3.2.0
• Upgrade asynciterator to v2.0.1

v1.4.3 - 2018-11-09

Changed

• Update to generic RDFJS typings
• Update N3 to 1.0.0, Closes #314

v1.4.2 - 2018-11-05

Fixed

• Pass baseIRI to RDF parsers, Closes #311

v1.4.1 - 2018-10-04

Fixed

• Make number mediator pick the first when all are invalid

v1.4.0 - 2018-10-03

Added

• Add MINUS operator actor
• Add preconfigured prefixes, Closes #80
• Also check extensions when no valid media type is present, Closes #286
• Make JSON-LD parser accept application/json
• Update sparqljson-parse to 1.4.0 with fix for Virtuoso, Closes #278

Fixed

• Fix federated actor stringifying sources, Closes #265
• Fix federation actor occasionally failing to produce results
• Fix crash when passing raw context with GraphQL, Closes #293
• Fallback to file-based sources when no content type is present, #286
• Fix parsing errors with datetime on resources without Memento support

1.3.0 - 2018-09-10

Added

• Add RDF/XML parser
• Enable serialization in actor-init-sparql-rdfjs

1.2.2 - 2018-09-05

Fixes

• Bump sparqljson-to-tree to 1.2.2

1.2.1 - 2018-09-05

Fixes

• Bump dependencies with invalid tslib dependency
• Add missing actor-query-operation-service dependencies
• Update peer dependency version

1.2.0 - 2018-09-05

Added

• Path expressions
• GraphQL-LD queries
• Memento support with the -d command line option
• SERVICE clause
• Dockerfile
• Browser script uploads to rdfjs/comunica-browser
• RDFJS sources
• Make browser version of the SPARQL init actor consistent with Node
• Allow initial bindings to be passed to query context
• Dereferenceable config instances
• Modularization of configs into config sets
• Action contexts
• Allow actions to be passively observed
• Optimize single SPARQL endpoint sources
• Set user agent for outgoing HTTP requests
• Add logger
• Add federated actor to default HDT init config

Fixes

• Allow pipeline mediator to have no actors in its bus
• Accept falsy terms in quad pattern RDF resolvers
• Fix falsy term crash when querying QPF interfaces
• Update asynciterator and follow-redirects to fix termination problems, Closes #144
• Make empty BGPs return a single empty bindings

1.1.2 - 2018-06-26

Added

• Created actor to read local files

Fixes

• Fix no client-side filtering of non-matching quads, Closes #152
• Update HDT to 2.x.x

1.1.1 - 2018-06-01

Fixes

• Fix config files missing from some init actor after publication

1.1.0 - 2018-06-01

Added

• Add --version flag to command, Closes #118
• Add Pipeline Combine mediator
• Add context preprocessing bus
• Add File RDF Source Identifier
• Add Hypermedia RDF Source identifier
• Add SPARQL RDF Source Identifier
• Add RDF Source Identifier actor as context preprocessor
• Add separate SPARQL init file actor
• Add separate SPARQL init HDT actor

Fixes

• Fix HTTP service not properly recognizing SPARQL POST content types
• Make spec-incompliant expression evaluator handle lang() more loosely
• Fix ActorHttpNative compilation errors on some platforms
• Don't emit null languages in basic expression evaluator
• Catch engine init errors in HTTP service
• Fix incorrect merge of FROM and FROM NAMED graphs
• Loosen handling of undefineds in expression evaluator
• Convert blank nodes when resolving SPARQL patterns, Closes #104
• Make N3 parser and serializer forward error events, #103
• Delegate stream errors from serializers to root, #103
• Make primary topic actor order-independent and take into account void:subset
• Fix paged RDF dereferencer not forwarding triples flag
• Bump Components.js to 2.2.0 to fix config compilation issues on Windows
• Prevent end event from firing before metadata

1.0.4 - 2018-03-22

Fixes

• Fix static newQuery function failing

1.0.3 - 2018-03-22

Fixes

• Catch failure to resolve federated metadata, #117

1.0.2 - 2018-03-22

Fixes

• Fix file quad resolver actor not included as dependency, Closes #120

1.0.1 - 2018-03-21

Fixes

• Add file quad pattern resolver to default config

[1.0.0] - 2018-03-19

• Initial release