Weird Development Ages on Sample Dataset when using pandas 2.0.2 #439
FlawlessNa
started this conversation in
General
Replies: 1 comment
-
Thanks, I haven't yet run our test suite against pandas>=2.0. Someone also just posted issue #438 referencing the same. Sounds like there are breaking changes that need to be addressed. This issue gives me a good place to start looking. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello,
First of all thanks for this great package! It is extremely useful!
I recently started using the
chainladder
package and first noticed an issue when combined withpandas
2.0.2.I'm not entirely sure whether the issue arises from
pandas
itself, or from the triangle creation withinchainladder
. This is why I decided to ask in the discussion section first.Note: I've ran the script under both Python 3.8 and 3.11 and got the exact same results.
Here is the script I am running to outline the issue (I've tried with a random triangle sampled from "clrd" and observed the same problem as well):
![image](https://private-user-images.githubusercontent.com/106719178/246151642-34fc4522-b795-49a2-8e3e-7b066005907b.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mzk1Nzk3NTQsIm5iZiI6MTczOTU3OTQ1NCwicGF0aCI6Ii8xMDY3MTkxNzgvMjQ2MTUxNjQyLTM0ZmM0NTIyLWI3OTUtNDlhMi04ZTNlLTdiMDY2MDA1OTA3Yi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwMjE1JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDIxNVQwMDMwNTRaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0yNzhmZWEwMjZiOWFjMDkxNTYwZGUxZTNiZDBlOGI0M2I2YzQyZTRkMmIyZWQzMDc3OWUzZGM3NTA5MmFjY2ZhJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.e5wboobSO-GaccIaF76NtGRWryFvyYz7C2SPVNyJo5k)
If I run this script with
![image](https://private-user-images.githubusercontent.com/106719178/246152520-424e3daf-2426-4032-8a9b-aaef35d367e3.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mzk1Nzk3NTQsIm5iZiI6MTczOTU3OTQ1NCwicGF0aCI6Ii8xMDY3MTkxNzgvMjQ2MTUyNTIwLTQyNGUzZGFmLTI0MjYtNDAzMi04YTliLWFhZWYzNWQzNjdlMy5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwMjE1JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDIxNVQwMDMwNTRaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT04ZTAyNDY5NDhiYWQyMzA3MTIzOWQ1OGJmY2M2NWMzNDBiNDU0M2E4MDUzNWE5ZGQ5NTExNmY1MmM5ZmFjMWYyJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.4dqcN9paR0G0jBLT7s_HQvFQlMa_jdVFFsdlCua65YI)
pandas
1.5.3 (similar version used in thechainladder
documentation), then everything goes smoothly, there are no problems with the output triangle, as below:Now if I run this script with
pandas 2.0.2
, notice what happens to the Development Ages (and similarly, thedata.valuation
dates end up being problematic as well)This is effectively a problem because it messes up the
cl.Development()
model fit when combined with various parameters (n_periods, drop_low/high, etc).I've tried digging to find the root of the discrepancy, and it looks like it first arises in
TriangleBase._development_lag
which is called when usingcl.load_sample.
I'm not sure of the best solution but I thought somebody might be able to help!
Thanks!!
Beta Was this translation helpful? Give feedback.
All reactions