Streaming systems are complex; apart from correct functionality (which might differ between implementations and vendors) many non-functional aspects can be benchmarked such as memory comsumption, latency, and throughput. For RDF Stream Processing several benchmarks exist, shown as follows. From data stream management, older benchmarks exist which are not specific to RDF data but might be adapted. Some are listed below.
RDF Stream Benchmarks
- SRBench – Streaming RDF/SPARQL Benchmark 1
- CSRBench – Correctness checking Benchmark for Streaming RDF/SPARQL 2
- LSBench – Linked Stream Benchmark 3
- YABench – Yet Another RDF Stream Processing Benchmark
Other Stream or CEP Benchmarks
- BEAST 4
- NEXMark 5
- Linear Road 6
- BiCEP 7 – a benchmarking framework
- Fast Flower Delivery (FFD) 8 – a functional benchmarking scenario
-
Zhang, Y.; Duc, P.; Corcho, O. & Calbimonte, J.-P. SRBench: A Streaming RDF/SPARQL Benchmark The Semantic Web –- ISWC 2012, Springer Berlin Heidelberg, 2012, 7649, 641-657 ↩
-
Dell’Aglio, D.; Calbimonte, J.-P.; Balduini, M.; Corcho, O. & Della Valle, E. On Correctness in RDF Stream Processor Benchmarking. The Semantic Web – ISWC 2013, Springer Berlin Heidelberg, 2013, 8219, 326-342 ↩
-
Le-Phuoc, D.; Dao-Tran, M.; Pham, M.-D.; Boncz, P.; Eiter, T. & Fink, M. Linked Stream Data Processing Engines: Facts and Figures. The Semantic Web – ISWC 2012, Springer Berlin Heidelberg, 2012, 7650, 300-312 ↩
-
Geppert, A.; Berndtsson, M.; Lieuwen, D. & Roncancio, C. Performance evaluation of object-oriented active database systems using the BEAST benchmark. Theor. Pract. Object Syst., John Wiley & Sons, Inc., 1998, 4, 135-149 ↩
-
Tucker, P.; Tufte, K.; Papadimos, V. & Maier, D. NEXMark – A benchmark for querying data streams. Oregon Health & Sciences University, 2002 ↩
-
Arasu, A.; Cherniack, M.; Galvez, E.; Maier, D.; Maskey, A. S.; Ryvkina, E.; Stonebraker, M. & Tibbetts, R. Linear road: a stream data management benchmark. VLDB ’04: Proceedings of the Thirtieth international conference on Very large data bases, VLDB Endowment, 2004, 480-491 ↩
-
Bizarro, P. BiCEP – Benchmarking Complex Event Processing Systems Event Processing, Internationales Begegnungs- und Forschungszentrum für Informatik (IBFI), Schloss Dagstuhl, Germany, 2007 ↩
-
Etzion, O. & Niblett, P. Event Processing in Action Manning Publications Co., 2010 ↩