This is a Python library that binds to Apache Arrow in-memory query engine
DataFusion.
DataFusion's Python bindings can be used as a foundation for building new data
systems in Python. Here are some examples:
- Dask SQL uses DataFusion's Python bindings for SQL parsing, query planning,
and logical plan optimizations, and then transpiles the logical plan to Dask
operations for execution.
- DataFusion Ballista is a distributed SQL query engine that extends
DataFusion's Python bindings for distributed use cases.
- DataFusion Ray is another distributed query engine that uses DataFusion's
Python bindings.
NOTE: If this package has multiple flavors (see below), then use one of them instead of the name specified above. NOTE: This is a Python port. Instead of py312-datafusion listed in the above command, you can pick from the names under the Packages section.
databases/py-datafusion: Update to 51.0.0
- Use USE_GITHUB temporarily when the python and rust crate has same version
It helps to avoid directory conflicts between datafusion-51.0.0 from PyPI
source distribution and Rust crate
Changes: https://github.com/apache/datafusion-python/tree/main/dev/changelog
databases/py-datafusion: Add py-datafusion 48.0.0
This is a Python library that binds to Apache Arrow in-memory query engine
DataFusion.
DataFusion's Python bindings can be used as a foundation for building new data
systems in Python. Here are some examples:
- Dask SQL uses DataFusion's Python bindings for SQL parsing, query planning,
and logical plan optimizations, and then transpiles the logical plan to Dask
operations for execution.
- DataFusion Ballista is a distributed SQL query engine that extends
DataFusion's Python bindings for distributed use cases.
- DataFusion Ray is another distributed query engine that uses DataFusion's
Python bindings.