What is the best approach to mass upload data from Excel to an application database?

Peer InsightsDocument Databases
772 viewscircle icon1 Upvotecircle icon4 Comments
Sort by:
Senior Data Scientist in Miscellaneousa year ago

Aside of the two possibilities already mentioned (CSVs from Excel, and Python), if you are sufficiently proficient R this might be a third option using its large set of different connectors to data sources.

Python and R also provide the possibility to do some data preparation and data validation in-between gathering data from Excel and uploading it into the database to ensure that the data types remain correct and formatting is not altered.

Lightbulb on1
Vice President in Bankinga year ago

I believe most standard databases out there support simple file upload option. Which one do you use ?

Lightbulb on1
Vice President, Information Technologya year ago

Python. Literally can be done with 2 lines of code, using something like pandas.

Lightbulb on1
IT Manager in Constructiona year ago

Hi, which DB are you thinking about? Anyway I believe the most important part is the Excel clean up and optimization then a right export usually in CSV and a table schema in the target DB which matches exactly the CSV.
You can try also via API: Google Sheets can serve as a lightweight database, and you can use the Google Sheets API to upload data: https://developers.google.com/sheets/api/guides/concepts

Content you might like

What are your organization’s drivers for implementing RegTech?

Support future growth36%

Automate manual processes59%

Demonstrate compliance49%

Reduce risk exposure43%

Improve customer experience16%

Reduce costs13%

View Results

How do you manage risk with contractors in your supply chain?

Keep hard copies and file paperwork13%

Update spreadsheets to create reports44%

Use a digital supply chain management solution27%

Partner with a third-party vendor11%

Other (comment below)3%

View Results

Does someone have advice on how they have balanced the risk/reward as it relates to introducing Open-Source Software to organization? My org is dipping its toe into the OSS world and the architecture and risk governance teams are looking to get ahead of these requests by coming up with policies and standards for OSS technology being brought into our environment. To clarify, this is not for software development and CI/CD pipelines, SDLC etc. This is for installing solutions that already exist out there (Zabbix, GreyLog, Prometheus etc.) into the organization's environment. We are looking to provide a balance on the obvious risk with the ability to move fast (like everyone else now).

What is a "must have" meeting for you or your team? If you can please share why/what makes it so great - topic, frequency, etc. - I'm looking to brainstorm some fresh opportunities for my group.

Read More Comments

We are looking for internal Audi systems options that fulfill our main requirements which are SaaS based hosted ONLY within the Saudi Arabia. The preferred solution should cover internal audit domain , subscription based and hosted inside Saudi Arabia to fulfill cybersecurity regulations. What are your recommendations?