Thank you again, Hareesh. In our testings, we have received files between 2M to 60M+ during testing. These files, according to the other team, are simulations of what will happen in production. So I am thinking of splitting the data by every 1000 lines regardless of the size(I cannot predict how big the file is). If the file is 60M+, they will have over 100K lines of data.
Regards,
Jonathan.