Currently, in my organization, we have BI products built on top of a data warehouse implemented using PostgreSQL, following a snowflake schema architecture (fact and dimension tables). This structure has become extremely complex and hard to maintain. Updates are very costly, and I’m heavily dependent on a single person to keep it running. I’m considering moving to a cloud-based environment, such as AWS Redshift or Google BigQuery. Based on your experience, would migrating to one of these platforms help me move away from this complex snowflake architecture? Also, between the two solutions (AWS Redshift and Google BigQuery), which one would you recommend and why?

Vendor/Product Recommendation Vendor/Product Assessment

1.8k views3 Comments

Sort by:

Principal Software Engineer, Data Engineering in Energy and Utilities2 months ago

It is better to analyze from the problem perspective. Whether you need a Snowflake schema, star schema, or one big table, it needs to be analyzed based on the user requirements, data volume, query patterns, business asks, latency requirements, etc. Changing the database/platforms might not solve the issue if you need a sustainable solution.

IT Manager in Banking3 months ago

Warehouse performance can be linked to multiple factors, like actual design vs blueprint design, how it has been built, underlying infrastructure and then comes technology choices. Before looking at a cloud data platform and thinking of moving to solve your problem, try to assess if your implemented design is scalable or not.

Data Manager3 months ago

Based on what you have described, it sounds like your problem is more the architecture or design of your database. I don't think changing the underlying database technology is going to address your problem.

Is your problem more related to the process you use to update your DW rather than the database itself?

What tools are you using to update your star / snowflake schema?

Content you might like

We are considering implementing SAP Fionner FPSL (Financial Products Subledger) in a banking environment and I'm looking to gather insights from those who have already gone through this journey. Key Questions: Implementation Experience: What has been your overall experience implementing SAP Fionner FPSL in a banking environment? What were the most significant challenges you faced? Lessons Learned: If you could restart your implementation, what would you do differently? Were there any particular pitfalls that could have been avoided with better planning? Success Criteria: What were your key success metrics for the implementation? How did you measure whether the project was successful? Cost Optimization: How did you manage to reduce integration costs within your banking ecosystem? What strategies proved most effective for keeping costs under control while maintaining functionality? Integration Challenges: What were the most difficult aspects of integrating FPSL with your existing banking systems? How did you overcome these challenges? Timeline Expectations: How long did your implementation take from planning to go-live? Were there any unexpected delays and how did you address them? Resource Requirements: What was the composition of your implementation team? What skills were most valuable during the implementation process?

Is Vendor Management part of Procurement in your organization?

Yes, it's integrated together49%

No, VMO is separated47%

Other (comment below)3%

View Results

Has anyone tried Telmai? Pros/cons? I find it a bit technical in nature.

What are three must-have advantages you look for when considering data integration service providers?

Easy to use unified view of data31%

Good data cleansing and transformation capabilities47%

Real-time or near real-time data integration54%

Above average automation capabilities31%

Works with all our applications and systems33%

Data governance support22%

Ability to scale and adapt to new data sources28%

Affordability of the solution18%

Potential for cost optimization10%

Other