No module named 'splink.duckdb'

[VaishnaviDhulipalla2902]

I am trying to use Splink for record linkage and the import itself throws an error No module named 'splink.duckdb'. I have installed splink using anaconda - conda install -c conda-forge splink . How do I resolve this?

[RobinL]

What do you get if you run

import splink
splink.__version__

[jbailiss]

Sorry to resurrect an old conversation but I am receiving the same error message.

I have previously used Splink successfully but have now migrated to a new server which required a reinstall. I am using Anaconda and installed using: conda install -c conda-forge splink

splink.version
4.0.6

duckdb.version
1.1.3

python_version()
3.12.3

from splink.duckdb.linker import DuckDBLinker
ModuleNotFoundError: No module named 'splink.duckdb'

I'm at a bit of a loss so any advice would be really appreciated!

Thanks

The logs from the install are below:

The following NEW packages will be INSTALLED:

libexpat conda-forge/win-64::libexpat-2.6.3-he0c23c2_0
libsqlite conda-forge/win-64::libsqlite-3.47.0-h2466b09_1
libzlib conda-forge/win-64::libzlib-1.2.13-h2466b09_6
python-duckdb conda-forge/win-64::python-duckdb-1.1.3-py312h275cf98_0
python_abi conda-forge/win-64::python_abi-3.12-5_cp312
splink conda-forge/noarch::splink-4.0.6-pyhd8ed1ab_0
sqlglot conda-forge/noarch::sqlglot-25.33.0-pyhd8ed1ab_0
ucrt conda-forge/win-64::ucrt-10.0.22621.0-h57928b3_1
vc14_runtime conda-forge/win-64::vc14_runtime-14.42.34433-he29a5d6_23

The following packages will be UPDATED:

openssl pkgs/main::openssl-3.0.15-h827c3e9_0 --> conda-forge::openssl-3.4.0-h2466b09_0
vs2015_runtime pkgs/main::vs2015_runtime-14.40.33807~ --> conda-forge::vs2015_runtime-14.42.34433-hdffcdeb_23
zlib pkgs/main::zlib-1.2.13-h8cc25b3_1 --> conda-forge::zlib-1.2.13-h2466b09_6

The following packages will be SUPERSEDED by a higher-priority channel:

certifi pkgs/main/win-64::certifi-2024.8.30-p~ --> conda-forge/noarch::certifi-2024.8.30-pyhd8ed1ab_0
conda pkgs/main::conda-24.11.0-py312haa9553~ --> conda-forge::conda-24.11.0-py312h2e8e312_0
python pkgs/main::python-3.12.7-h14ffc60_0 --> conda-forge::python-3.12.3-h2628c8c_0_cpython

[RobinL]

I think your code was written for Splink 3, so you need to install to https://pypi.org/project/splink/3.9.15/ or upgrade your code to splink 4 syntax https://moj-analytical-services.github.io/splink/blog/2024/07/24/splink-400-released.html

[jbailiss]

Thanks Robin. That was indeed the cause.

The conversion guide is very helpful and all is working on the new server now.

[py67]

I am facing a similar issue - can't import splink. I am using conda environment and list conda shows splink installed in my conda environment. Below is the message I get in importing splink.

import splink
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\yadavpk\Anaconda3\envs\nsarV\Lib\site-packages\splink_init_.py", line 4, in
from pyspark.sql.dataframe import DataFrame
File "C:\Users\yadavpk\Anaconda3\envs\nsarV\Lib\site-packages\pyspark_init_.py", line 51, in
from pyspark.context import SparkContext
File "C:\Users\yadavpk\Anaconda3\envs\nsarV\Lib\site-packages\pyspark\context.py", line 31, in
from pyspark import accumulators
File "C:\Users\yadavpk\Anaconda3\envs\nsarV\Lib\site-packages\pyspark\accumulators.py", line 97, in
from pyspark.serializers import read_int, PickleSerializer
File "C:\Users\yadavpk\Anaconda3\envs\nsarV\Lib\site-packages\pyspark\serializers.py", line 71, in
from pyspark import cloudpickle
File "C:\Users\yadavpk\Anaconda3\envs\nsarV\Lib\site-packages\pyspark\cloudpickle.py", line 145, in
_cell_set_template_code = _make_cell_set_template_code()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\yadavpk\Anaconda3\envs\nsarV\Lib\site-packages\pyspark\cloudpickle.py", line 126, in _make_cell_set_template_code
return types.CodeType(
^^^^^^^^^^^^^^^
TypeError: code expected at least 16 arguments, got 15

I can see splink installed in conda environment nsarV (below is what I get):

(nsarV) PS C:\Users\yadavpk> conda list splink

packages in environment at C:\Users\yadavpk\Anaconda3\envs\nsarV:

Name Version Build Channel

splink 1.0.6 pyhd8ed1ab_0 conda-forge

Thanks!

[RobinL]

Hmm, not sure. Splink 1.0.6 is a very old version of Splink. I asked chat GPT 4 and it said:

The error message "TypeError: code expected at least 16 arguments, got 15" suggests that there's a version mismatch somewhere. It seems like you are running a version of PySpark that's incompatible with the Python version you have installed in your conda environment.

PySpark is a Python library for Apache Spark, a big data processing engine. It seems like the version of PySpark being used by Splink is not compatible with your Python version.

Is that a possibility?

[py67]

Your reply made me notice that the version of Splink I recently installed using Conda is indeed very old. Just installed version 3.8.1 and the problem is gone. Thanks so much!