Tech Talk
When The Data Get Big(ger) Part2
It is crucial to move beyond using *.CSV files when the scope of a project goes beyond 1MM rows of data. This is an add-on to my initial post about using Pyarrow Feather in Python for efficient data storage. The first post makes a good case for this, but it leaves out some nitty gritty details…
When the Data Get Big(ger)
This blog describes how we manage life when the data get too big for Excel-compatible formats such as CSV. Practically, when working with exploratory data analysis, there is no rush to jump to fancy data formats on the client-facing side of things. Even when people hire a consultant to help with…