diff --git a/doc/CLAUDE.md b/doc/CLAUDE.md index e3b074940..2189821d8 100644 --- a/doc/CLAUDE.md +++ b/doc/CLAUDE.md @@ -17,7 +17,7 @@ doc/ │ ├── first_plot.qmd Tutorial: first visualization │ ├── grammar.qmd Grammar of Graphics conceptual foundation │ ├── anatomy.qmd Anatomy of a ggsql query -│ ├── tooling.qmd VS Code / Positron / Jupyter / Python / R / CLI integrations +│ ├── tooling/ VS Code / Positron / Jupyter / Python / R / CLI / Wasm │ └── the_rest.qmd Advanced features ├── syntax/ │ ├── index.qmd diff --git a/doc/_quarto.yml b/doc/_quarto.yml index 0ba2573cc..880752e1d 100644 --- a/doc/_quarto.yml +++ b/doc/_quarto.yml @@ -139,8 +139,22 @@ website: href: get_started/grammar.qmd - text: Anatomy of ggsql href: get_started/anatomy.qmd - - text: Tooling - href: get_started/tooling.qmd + - section: Tooling + contents: + - text: Overview + href: get_started/tooling/index.qmd + - text: Positron / VS Code + href: get_started/tooling/positron-vscode.qmd + - text: Jupyter & Quarto + href: get_started/tooling/jupyter-quarto.qmd + - text: CLI + href: get_started/tooling/cli.qmd + - text: Python + href: get_started/tooling/python.qmd + - text: R + href: get_started/tooling/r.qmd + - text: WebAssembly + href: get_started/tooling/webassembly.qmd - text: The rest of the owl href: get_started/the_rest.qmd - text: Wrap up diff --git a/doc/assets/logos/javascript.svg b/doc/assets/logos/javascript.svg new file mode 100644 index 000000000..9650ca78e --- /dev/null +++ b/doc/assets/logos/javascript.svg @@ -0,0 +1,4 @@ + + + + \ No newline at end of file diff --git a/doc/assets/logos/webassembly.svg b/doc/assets/logos/webassembly.svg new file mode 100644 index 000000000..f2d67d77a --- /dev/null +++ b/doc/assets/logos/webassembly.svg @@ -0,0 +1,8 @@ + + + + + + + + \ No newline at end of file diff --git a/doc/get_started/installation.qmd b/doc/get_started/installation.qmd index b40863127..a2e3578af 100644 --- a/doc/get_started/installation.qmd +++ b/doc/get_started/installation.qmd @@ -284,3 +284,16 @@ ggsql-jupyter --install For syntax highlighting and language support in VS Code or Positron, install the ggsql extension. You can either install it directly from the [extension marketplace](https://open-vsx.org/extension/ggsql/ggsql) from within the IDE or download and install it manually (in the *Extensions* view, click the `...` menu, select "Install from VSIX...", and choose the downloaded file.) ::: + +::: {.icon-section} +[![](../assets/logos/webassembly.svg){height=60} ![](../assets/logos/javascript.svg){height=60}]{.section-icons} + +## WebAssembly / JavaScript + +To use ggsql in a web application or JavaScript environment, you can install the `ggsql-wasm` package from [npm](https://www.npmjs.com/package/ggsql-wasm): + +```bash +npm install ggsql-wasm +``` + +::: diff --git a/doc/get_started/tooling.qmd b/doc/get_started/tooling.qmd deleted file mode 100644 index c361c1f02..000000000 --- a/doc/get_started/tooling.qmd +++ /dev/null @@ -1,50 +0,0 @@ ---- -title: Tooling ---- - -Now that we understand some of the most important parts of the syntax let's spend a bit of time on where and how to apply it. All the examples on this page are interactive and runs directly in the browser, which is obviously useful for teaching, but it will not suffice for your day-to-day work where you need to interact with your own data. ggsql is a general tool you can use in a multitude of ways and we'll go over the most important below. - -## VS Code extension -We provide an extension for VS Code/Positron that brings language support to the IDE. Positron is generally superior for data analysis and the ggsql integration is deeper there, which we will showcase below. Still, using the extension with VS Code should provide you with a good developer experience. You can grab the [ggsql extension](https://open-vsx.org/extension/ggsql/ggsql) directly from the marketplace. - -Once installed you will get access to ggsql as a language at the same level as R and Python. You can open and edit `.gsql` files with syntax highlighting, autocomplete, you can open up a REPL in the console pane and executing queries and you can see the resulting visualization appear in the plot pane. If you have any database connections in the connection pane you can directly attach these to your ggsql runtime and begin to visualize the tables in there. - -## Jupyter kernel -Once the Jupyter kernel is installed you can use ggsql as an engine in your Jupyter notebooks and Quarto documents. For a Jupyter notebook you can select the kernel when you start a new notebook. For a Quarto document you use the ggsql language name to tell the renderer to use the ggsql kernel e.g. - -```{{ggsql}} -VISUALISE ... -``` - -Each block in the document uses the same session, so tables created in one block will be available in subsequent blocks. - -## Python package -We have a [python package](https://pypi.org/project/ggsql/) which you can install through pip (`pip install ggsql`). The package provides binding to ggsql and allows you to plot with ggsql directly from within python and register alternative data backends. - -A simple example could be - -```python -import ggsql -import polars as pl - -# Create a DataFrame -df = pl.DataFrame({ - "x": [1, 2, 3, 4, 5], - "y": [10, 20, 15, 30, 25], - "category": ["A", "B", "A", "B", "A"] -}) - -# Render to Altair chart -chart = ggsql.render_altair(df, "VISUALISE x, y DRAW point") - -# Display or save -chart.display() # In Jupyter -chart.save("chart.html") # Save to file -``` - -## Command line interface -While maybe not the most ergonomic way to interact directly with ggsql, there is a CLI interface if you need to build tools around ggsql. The CLI tool allows you to execute a file or string and validate a query without executing it. A simple example of executing a query looks like this: - -```bash -ggsql --exec "VISUALISE species AS fill FROM ggsql:penguins DRAW bar" -``` diff --git a/doc/get_started/tooling/cli.qmd b/doc/get_started/tooling/cli.qmd new file mode 100644 index 000000000..f3a99e158 --- /dev/null +++ b/doc/get_started/tooling/cli.qmd @@ -0,0 +1,87 @@ +--- +title: Command line interface +--- + +The `ggsql` command line interface lets you execute queries and validate syntax directly from the terminal. While it may not be the most ergonomic way to interact directly with ggsql, it is useful for scripting, automation, and building tools around ggsql. + +## Installation + +Install the ggsql CLI on your system using the standard [installation instructions](../installation.html). + +## Executing a query + +You can execute a ggsql query from a string with `ggsql exec`: + +```bash +ggsql exec "VISUALISE species AS fill FROM ggsql:penguins DRAW bar" +``` + +Or run a `.gsql` file: + +```bash +ggsql run my_query.gsql +``` + +In both cases, output is written to `stdout` as a Vega-Lite JSON spec. You can redirect it to a file: + +```bash +ggsql run my_query.gsql > chart.vl.json +``` + +Such files can be rendered as images using tools that work with Vega-Lite specs, such as the [Online Vega Editor](https://vega.github.io/editor/) or [vl-convert](https://github.com/vega/vl-convert) command line tool. For example, + +```bash +vl-convert vl2png -i chart.vl.json -o chart.png +``` + +A standard SQL query can also be provided to ggsql. If the query returns a table, the resulting values will be written to `stdout`. + +## Validating a query + +If you only want to check that a query is syntactically valid without executing it, use `ggsql validate`: + +```bash +$ ggsql validate "VISUALISE x, y FROM table DRAW point" +✓ Query syntax is valid +``` + +## Database connections + +Both `ggsql exec` and `ggsql run` accept a `--reader` flag that can be used to specify a connection string to be used when executing the query. If not provided, ggsql will use an empty in-memory duckdb connection, equivalent to `--reader duckdb://memory`. + +```bash +$ ggsql exec --reader sqlite://sample/ggsql_test.sqlite \ + "SELECT * FROM test_table LIMIT 3" +col_a, col_b, col_c +215.87, 75.11, delta +418.78, 71.75, delta +495.75, 12.55, delta + +$ ggsql exec --reader odbc://DSN=ggsql-pg-test \ + "SELECT * FROM test_table LIMIT 3" +col_a, col_b, col_c +319.34, 91.45, gamma +299.08, 49.36, epsilon +12.5, 29.48, gamma +``` + +## Documentation + +The ggsql CLI has built-in documentation for ggsql syntax and usage. Run `ggsql docs` for an overview of available documentation topics, and `ggsql docs [topic]` to read about a specific topic. + +```bash +$ ggsql docs draw +DRAW is perhaps the most important clause in ggsql as it defines a layer in your +visualisation. A layer is a single instance of a visual representation of a dataset. +[...] +``` + +A ggsql skill, a usage guide intended for AI assistants and humans, can also be output using the `ggsql skill` command. + +```bash +$ ggsql skill + ggsql Query Writer +ggsql is a SQL extension for declarative data visualization based on +Grammar of Graphics principles. +[...] +``` diff --git a/doc/get_started/tooling/index.qmd b/doc/get_started/tooling/index.qmd new file mode 100644 index 000000000..20f9e4682 --- /dev/null +++ b/doc/get_started/tooling/index.qmd @@ -0,0 +1,17 @@ +--- +title: Tooling +--- + +Now that we understand some of the most important parts of the syntax let's spend a bit of time on where and how to apply it. All the examples on this website are interactive and run directly in the browser, which is obviously useful for teaching, but it will not suffice for your day-to-day work where you need to interact with your own data. ggsql is a general tool you can use in a multitude of ways and we'll go over them in this section. + +- **[Positron / VS Code](positron-vscode.qmd)** --- Full language support in your IDE with syntax highlighting, autocomplete, and an integrated REPL. + +- **[Jupyter & Quarto](jupyter-quarto.qmd)** --- Use ggsql as a kernel in Jupyter or Quarto for interactive data exploration and reproducible documents. + +- **[CLI](cli.qmd)** --- Execute queries and validate syntax from the command line. + +- **[Python](python.qmd)** --- Use ggsql from Python. + +- **[R](r.qmd)** --- Use ggsql from R. + +- **[WebAssembly](webassembly.qmd)** --- Run ggsql entirely in the browser with no installation. diff --git a/doc/get_started/tooling/jupyter-quarto.qmd b/doc/get_started/tooling/jupyter-quarto.qmd new file mode 100644 index 000000000..872f18574 --- /dev/null +++ b/doc/get_started/tooling/jupyter-quarto.qmd @@ -0,0 +1,66 @@ +--- +title: Jupyter & Quarto +--- + +Once the ggsql Jupyter kernel has been installed you can use ggsql in Jupyter notebooks and Quarto documents. + +## Installation + +First, install ggsql on your system using the Jupyter kernel [installation instructions](../installation.html). If you do not already have Jupyter installed, you should install it in a new virtual environment using [uv](https://docs.astral.sh/uv/) or `pip` first: + +```bash +uv venv +uv pip install jupyter +uv pip install ggsql-jupyter +source .venv/bin/activate +ggsql-jupyter --install +``` + +To verify installation, run `jupyter kernelspec list` and check that `ggsql` is listed as an available kernel. + +```bash +$ jupyter kernelspec list +Available kernels: + python3 /tmp/.venv/share/jupyter/kernels/python3 + ggsql /home/user/.local/share/jupyter/kernels/ggsql +``` + +You can now use ggsql in your Quarto documents or Jupyter notebooks. + +## Quarto documents + +For a Quarto document, use the `ggsql` language name in a code block to tell the renderer to use the ggsql kernel: + +````markdown +```{{ggsql}} +VISUALISE species AS fill +FROM ggsql:penguins +DRAW bar +``` +```` + +Each block in the document shares the same session, so tables created in one block will be available in subsequent blocks. + +![](./screenshots/quarto-document.png) + +## Jupyter notebooks + +First, start a Jupyter notebook server in your terminal: + +```bash +jupyter notebook +``` + +When creating a new notebook, select the ggsql kernel from the kernel selector. Each cell in the notebook can contain a ggsql query, and the resulting visualisation will be displayed inline below the cell. + +As with Quarto, each cell in the notebook uses the same session, so tables created in one cell will be available in subsequent cells. + +![](./screenshots/jupyter-notebook.png) + +## Database connections + +By default, the ggsql kernel starts with an empty in-memory duckdb database connection. A "magic" comment can be used to initiate a different database connection after the session has launched, which can be either a comment in your Quarto code block or invoked directly in a Jupyter notebook cell. + +The connection syntax is `-- @connect: [connection_uri]`, where `[connection_uri]` is a ggsql database connection string in the same format as accepted by the [ggsql CLI](cli.html). + +![](./screenshots/jupyter-connections.png) diff --git a/doc/get_started/tooling/positron-vscode.qmd b/doc/get_started/tooling/positron-vscode.qmd new file mode 100644 index 000000000..9eecc7112 --- /dev/null +++ b/doc/get_started/tooling/positron-vscode.qmd @@ -0,0 +1,54 @@ +--- +title: Positron / VS Code +--- + +We provide an extension for [VS Code](https://code.visualstudio.com) and [Positron](https://positron.posit.co/) that brings ggsql language support to the IDE. Positron is generally superior for data analysis and the ggsql integration is deeper there, which we will showcase below. Still, using the extension with VS Code should provide you with a good developer experience. Once installed you will get access to ggsql as a language at the same level as R and Python. You can open and edit `.gsql` files with syntax highlighting and autocomplete. + +In Positron you can open up a REPL in the console pane, execute queries, and see the resulting visualization appear in the plot pane. You can add database connections in the connection pane and directly attach these to your ggsql runtime and begin to visualize the tables in there. + +![](./screenshots/positron-ggsql-integration.png) + +## Installation + +First, install ggsql on your system using the [installation instructions](../installation.html). + +Then, you can grab the ggsql extension directly from the extension section of the Positron IDE, or download it manually from the [OpenVSX extension marketplace](https://open-vsx.org/extension/ggsql/ggsql) (to install, in the *Extensions* view click the `...` menu, select "Install from VSIX...", and choose the downloaded file.) + +![](./screenshots/positron-extensions-pane.png) + +## Console and plotting + +Once the extension has been installed, ggsql will be available to launch as a language runtime in the same way as an R or Python session in Positron's console session selector. + +![](./screenshots/positron-console-session.png) + +Once started, a ggsql session will be made available in the Console pane, providing a REPL for executing standard SQL or ggsql code. When first launched, ggsql will be started with an empty in-memory duckdb database connection. + +![](./screenshots/positron-console-pane.png) + +Executing ggsql statements containing [`VISUALISE`](../../syntax/clause/visualise.html) will render the resulting visualization in the Plots pane. + +![](./screenshots/positron-plots-pane.png) + + +## Editor and cells + +The `ggsql` language type can be used to enable syntax highlighting and autocomplete. Files with extension `.gsql` will automatically be configured to use the `ggsql` language type. + +In Positron, a "Source Current File" button is provided in the top left of the Editor pane, which will execute the entire contents of the currently open file in the Positron console. You can also create "cells" in your editor, which are blocks of code that can be independently executed. + +To create a cell, simply add a line with `-- %%` between blocks of code in your ggsql source file. You can then execute the contents a particular cell in the console by clicking the "Run Query" button that appears above the separator. + +![](./screenshots/positron-cells.png) + +## Database connections + +By default, ggsql starts in the Positron console with an empty in-memory duckdb database connection. A "magic" comment can be used to initiate a different database connection after the session has launched, which can be either a comment in your source file or invoked directly in the console. + +The connection syntax is `-- @connect: [connection_uri]`, where `[connection_uri]` is a ggsql database connection string in the same format as accepted by the [ggsql CLI](cli.html). + +![](./screenshots/positron-console-connections.png) + +The ggsql extension also integrates with Positron's connections pane, allowing you to easily connect to databases that you have configured in Positron. Connections made in the console, as above, will be saved to the connections pane automatically, or a connections wizard can be launched to help connect to supported databases by selecting the "New Connection" button in the Positron Connections pane. + +![](./screenshots/positron-connections.png) diff --git a/doc/get_started/tooling/python.qmd b/doc/get_started/tooling/python.qmd new file mode 100644 index 000000000..a7d843bed --- /dev/null +++ b/doc/get_started/tooling/python.qmd @@ -0,0 +1,35 @@ +--- +title: Python +--- + +We have a [Python package](https://pypi.org/project/ggsql/) that provides bindings to ggsql, allowing you to plot with ggsql directly from within Python and register alternative data backends. Install it with pip: + +```bash +pip install ggsql +``` + +## Basic usage + +The `ggsql` package can render a ggsql query against a DataFrame and return an Altair chart object: + +```python +import ggsql +import polars as pl + +df = pl.DataFrame({ + "x": [1, 2, 3, 4, 5], + "y": [10, 20, 15, 30, 25], + "category": ["A", "B", "A", "B", "A"] +}) + +chart = ggsql.render_altair(df, "VISUALISE x, y DRAW point") + +chart.display() # In Jupyter +chart.save("chart.html") # Save to file +``` + +The `render_altair()` function accepts both Polars and pandas DataFrames. + +## Further reading + +Full API documentation for the Python package is available at [python.ggsql.org](http://python.ggsql.org). diff --git a/doc/get_started/tooling/r.qmd b/doc/get_started/tooling/r.qmd new file mode 100644 index 000000000..138379880 --- /dev/null +++ b/doc/get_started/tooling/r.qmd @@ -0,0 +1,41 @@ +--- +title: R +--- + +The [ggsql R package](https://r.ggsql.org) provides bindings to ggsql, allowing you to run ggsql queries directly from R and render visualisations from data frames. It also contains a knitr engine for supporting ggsql blocks, with facilities for bidirectional data flow between R, Python, and ggsql blocks. + +## Installation + +ggsql can be installed from CRAN with: + +```r +pak::pak("ggsql") +``` + +To get the development version, change "ggsql" to "posit-dev/ggsql". + +## Basic usage + +When you load ggsql into R in your Rmarkdown/Quarto document the knitr engine is registered. This allows you to use ggsql directly in Rmarkdown and Quarto documents. + +While one of the core appeals of the ggsql R package is the knitr engine it provides, you can also use it directly in R to execute visual queries: + +```r +library(ggsql) + +# Create an in-memory DuckDB reader +reader <- duckdb_reader() + +# Register a dataset in it +ggsql_register(reader, mtcars, "amazing_data") + +# Visualize it with a query +ggsql_execute(reader, " +VISUALIZE mpg AS x, disp AS y FROM amazing_data +DRAW point +") +``` + +## Further reading + +Full documentation for the R package is available at [r.ggsql.org](https://r.ggsql.org). diff --git a/doc/get_started/tooling/screenshots/jupyter-connections.png b/doc/get_started/tooling/screenshots/jupyter-connections.png new file mode 100644 index 000000000..94fc79686 Binary files /dev/null and b/doc/get_started/tooling/screenshots/jupyter-connections.png differ diff --git a/doc/get_started/tooling/screenshots/jupyter-notebook.png b/doc/get_started/tooling/screenshots/jupyter-notebook.png new file mode 100644 index 000000000..93865dca6 Binary files /dev/null and b/doc/get_started/tooling/screenshots/jupyter-notebook.png differ diff --git a/doc/get_started/tooling/screenshots/positron-cells.png b/doc/get_started/tooling/screenshots/positron-cells.png new file mode 100644 index 000000000..37b9cdcb6 Binary files /dev/null and b/doc/get_started/tooling/screenshots/positron-cells.png differ diff --git a/doc/get_started/tooling/screenshots/positron-connections.png b/doc/get_started/tooling/screenshots/positron-connections.png new file mode 100644 index 000000000..00d14654b Binary files /dev/null and b/doc/get_started/tooling/screenshots/positron-connections.png differ diff --git a/doc/get_started/tooling/screenshots/positron-console-connections.png b/doc/get_started/tooling/screenshots/positron-console-connections.png new file mode 100644 index 000000000..804fd4493 Binary files /dev/null and b/doc/get_started/tooling/screenshots/positron-console-connections.png differ diff --git a/doc/get_started/tooling/screenshots/positron-console-pane.png b/doc/get_started/tooling/screenshots/positron-console-pane.png new file mode 100644 index 000000000..c324299f5 Binary files /dev/null and b/doc/get_started/tooling/screenshots/positron-console-pane.png differ diff --git a/doc/get_started/tooling/screenshots/positron-console-session.png b/doc/get_started/tooling/screenshots/positron-console-session.png new file mode 100644 index 000000000..48c57afc1 Binary files /dev/null and b/doc/get_started/tooling/screenshots/positron-console-session.png differ diff --git a/doc/get_started/tooling/screenshots/positron-extensions-pane.png b/doc/get_started/tooling/screenshots/positron-extensions-pane.png new file mode 100644 index 000000000..64f06af81 Binary files /dev/null and b/doc/get_started/tooling/screenshots/positron-extensions-pane.png differ diff --git a/doc/get_started/tooling/screenshots/positron-ggsql-integration.png b/doc/get_started/tooling/screenshots/positron-ggsql-integration.png new file mode 100644 index 000000000..525545ea5 Binary files /dev/null and b/doc/get_started/tooling/screenshots/positron-ggsql-integration.png differ diff --git a/doc/get_started/tooling/screenshots/positron-plots-pane.png b/doc/get_started/tooling/screenshots/positron-plots-pane.png new file mode 100644 index 000000000..54153d921 Binary files /dev/null and b/doc/get_started/tooling/screenshots/positron-plots-pane.png differ diff --git a/doc/get_started/tooling/screenshots/quarto-document.png b/doc/get_started/tooling/screenshots/quarto-document.png new file mode 100644 index 000000000..b75bcc67c Binary files /dev/null and b/doc/get_started/tooling/screenshots/quarto-document.png differ diff --git a/doc/get_started/tooling/webassembly.qmd b/doc/get_started/tooling/webassembly.qmd new file mode 100644 index 000000000..335a5afaf --- /dev/null +++ b/doc/get_started/tooling/webassembly.qmd @@ -0,0 +1,47 @@ +--- +title: WebAssembly +--- + +ggsql compiles to WebAssembly, which means it can run entirely in the browser with no server-side component. The [playground](/wasm/index.html) on this site is powered by the WebAssembly build, and all the interactive examples in the documentation run the same way. + +## Playground + +The easiest way to try ggsql without installing anything is the [online playground](/wasm/index.html). It provides an editor with syntax highlighting and a live preview of the resulting visualisation. Queries execute locally in your browser and no data leaves your machine. + +## Usage + +The WebAssembly build can be embedded in your own web pages using the [ggsql-wasm](https://www.npmjs.com/package/ggsql-wasm) npm package. This allows you to build custom interfaces and applications on top of ggsql running in the browser. + +Add ggsql-wasm to your project with: + +```bash +npm install ggsql-wasm +``` + +Then you can import the module and use it in your JavaScript code: + +```javascript +import init, { GgsqlContext } from "ggsql-wasm"; + +// Initialize the WebAssembly module and create a ggsql context +export const WASM_BASE = new URL(".", import.meta.url).href; +await init(WASM_BASE + "ggsql_wasm_bg.wasm"); +const context = new GgsqlContext(); + +// Fetch and register a CSV file as a table +const response = await fetch("sample_data.csv"); +const buffer = await response.arrayBuffer(); +const data = new Uint8Array(buffer); +context.register_csv('sample_data', data); + +// Execute a ggsql query against the registered data +const query = ` + VISUALISE x, y FROM sample_data + DRAW point +`; +const result = context.execute(query); + +// Use the result (a Vega-Lite spec) to render a chart, +// or log it to the JavaScript console +console.log(result); +```