Multiple warning about incompatible dtype casting, probably related to nan check

[AndhikaWB]

Version check:

Python 3.11.5, dython 0.7.5

Describe the bug:

• Multiple FutureWarning about incompatible dtype casting when using mixed association matrix
• The categorical/nominal columns on this dataset are already label encoded (as integer), but even when using astype('category') this warning still appear, I think your script is assuming something wrong about the data type, hence the warning appear
• There is no nan/null value on this dataset, and this warning can be annoying since I have many columns. Disabling warning may work, but I don't want to do that

Code to reproduce:

import dython
import pandas as pd

df = pd.read_csv('data/data.csv', sep = ';')

temp = df.replace({'Status': {
    'Dropout': 0,
    'Enrolled': 1,
    'Graduate': 2
}})

cat_col = ['Application_mode', 'Course', 'Fathers_occupation', 'Fathers_qualification', 'Marital_status', 'Mothers_occupation', 'Mothers_qualification', 'Nacionality', 'Previous_qualification']

temp = associations(
    temp,
    nominal_columns = cat_col, # + ['Status']
    nom_nom_assoc = 'theil',
    compute_only = True
)['corr']

temp

Error message:

Error message:

c:\Users\Dhika\Documents\Projects\StudentPerf\.venv\Lib\site-packages\dython\nominal.py:737: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  inf_nan.loc[columns[j], columns[i]] = _inf_nan_str(ji)
c:\Users\Dhika\Documents\Projects\StudentPerf\.venv\Lib\site-packages\dython\nominal.py:736: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  inf_nan.loc[columns[i], columns[j]] = _inf_nan_str(ij)
c:\Users\Dhika\Documents\Projects\StudentPerf\.venv\Lib\site-packages\dython\nominal.py:736: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  inf_nan.loc[columns[i], columns[j]] = _inf_nan_str(ij)
c:\Users\Dhika\Documents\Projects\StudentPerf\.venv\Lib\site-packages\dython\nominal.py:736: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  inf_nan.loc[columns[i], columns[j]] = _inf_nan_str(ij)
c:\Users\Dhika\Documents\Projects\StudentPerf\.venv\Lib\site-packages\dython\nominal.py:736: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  inf_nan.loc[columns[i], columns[j]] = _inf_nan_str(ij)
c:\Users\Dhika\Documents\Projects\StudentPerf\.venv\Lib\site-packages\dython\nominal.py:736: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  inf_nan.loc[columns[i], columns[j]] = _inf_nan_str(ij)
c:\Users\Dhika\Documents\Projects\StudentPerf\.venv\Lib\site-packages\dython\nominal.py:736: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  inf_nan.loc[columns[i], columns[j]] = _inf_nan_str(ij)
c:\Users\Dhika\Documents\Projects\StudentPerf\.venv\Lib\site-packages\dython\nominal.py:736: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  inf_nan.loc[columns[i], columns[j]] = _inf_nan_str(ij)
c:\Users\Dhika\Documents\Projects\StudentPerf\.venv\Lib\site-packages\dython\nominal.py:736: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  inf_nan.loc[columns[i], columns[j]] = _inf_nan_str(ij)
c:\Users\Dhika\Documents\Projects\StudentPerf\.venv\Lib\site-packages\dython\nominal.py:736: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  inf_nan.loc[columns[i], columns[j]] = _inf_nan_str(ij)
c:\Users\Dhika\Documents\Projects\StudentPerf\.venv\Lib\site-packages\dython\nominal.py:736: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  inf_nan.loc[columns[i], columns[j]] = _inf_nan_str(ij)
c:\Users\Dhika\Documents\Projects\StudentPerf\.venv\Lib\site-packages\dython\nominal.py:736: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  inf_nan.loc[columns[i], columns[j]] = _inf_nan_str(ij)
c:\Users\Dhika\Documents\Projects\StudentPerf\.venv\Lib\site-packages\dython\nominal.py:736: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.

... (many more lines with exact same warning)

Input data:

From here

[shakedzy]

Hey @AndhikaWB , as seen in the logs - the warnings originate from Pandas. Which version do you use?

[AndhikaWB]

@shakedzy v2.2.2, which I believe is the latest

[shakedzy]

I see, I'll take a look

[AndhikaWB]

Here's the real source of the data, in case the context is needed: https://www.kaggle.com/datasets/thedevastator/higher-education-predictors-of-student-retention

The source I gave above only renamed the column name (I think)

[shakedzy]

@AndhikaWB there's a fix ready on a branch: https://github.com/shakedzy/dython/tree/162-multiple-warning-about-incompatible-dtype-casting-probably-related-to-nan-check
Care to try and see if this solved the problem?

[shakedzy]

Fix merged, version 0.7.6