# Connecting PySpark via JDBC using Jupyter Notebook

**URL:** <https://community.cdata.com/t/connecting-pyspark-via-jdbc-using-jupyter-notebook/858>\
**Category:** CData Virtuality \[Archived\]\
**Tags:** article, cdata-virtuality\
**Created:** [July 12, 2024, 5:35pm UTC](https://community.cdata.com/t/connecting-pyspark-via-jdbc-using-jupyter-notebook/858 "2024-07-12T17:35:05Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![anonymous](https://avatars.discourse-cdn.com/v4/letter/a/8491ac/32.png) [@anonymous](https://community.cdata.com/u/anonymous)\
**Post date:** [July 12, 2024, 5:35pm UTC](https://community.cdata.com/t/connecting-pyspark-via-jdbc-using-jupyter-notebook/858/1 "2024-07-12T17:35:05Z")

</div>

# Install DataVirtuality JDBC driver

## Windows

Download the latest DataVirtuality JDBC driver from&nbsp;[here](https://datavirtuality.com/en/download-driver/).  
Copy the jar file to your spark install folder:

> ```auto
> D:\spark-2.3.1-bin-hadoop2.6\jars
> 
> ```

## Linux

Download the latest DataVirtuality JDBC driver from&nbsp;[here](https://datavirtuality.com/en/download-driver/).  
Copy the jar file to your spark install folder:

> ```auto
> ~/.local/lib/python3.11/site-packages/pyspark/jars/
> 
> ```

# Alternative solution using SparkConf

[https://stackoverflow.com/questions/46925864/how-to-add-jdbc-drivers-to-classpath-when-using-pyspark](https://stackoverflow.com/questions/46925864/how-to-add-jdbc-drivers-to-classpath-when-using-pyspark)

# Enabling JDBC connection

In your DataVirtuality web Interface go to Preferences, and make sure that JDBC port is open, and write down the port number.

 ![10644145395613](https://us1.discourse-cdn.com/cdata/original/1X/e323ac3e0e1eeca6db1707ca1665e8fcba80a7a6.png)

 ![10644139659677](https://us1.discourse-cdn.com/cdata/original/2X/c/c7710a48071f076fb893cbae86c6b0196837ddec.png)

# Querying from PySpark in Jupyter Notebook

Now that we have set up the JDBC driver we can connect to it and query data from it.

Open a new Jupyter Notebook session and copy the following

Replace the highlights with your own credentials, and provide a working sql query.

**NOTE:** &nbsp;&nbsp;use&nbsp; **mm** &nbsp;instead of&nbsp; **mms** &nbsp;in jdbc\_url, if you are not using a SSL connection.

> ```auto
> import findsparkfindspark.init()import pysparkfrom pyspark.sql import SparkSessionspark = SparkSession.builder.getOrCreate()from pyspark.sql import SparkSessionjdbc_url = "jdbc:datavirtuality:datavirtuality@mms://DVHostname:PortNumber;user=db_username;password=db_password"query = "(select * from hands_on.orders) as table1"df1 = spark.read.format('jdbc').options(driver = 'com.datavirtuality.dv.jdbc.Driver',url=jdbc_url, dbtable=query ).load()df1.show()
> 
> ```

 ![10644292186653](https://us1.discourse-cdn.com/cdata/original/2X/e/ecad54d3412420c2ed46f2e11c57b074e99d22b7.png)