Loading...
Answers
MenuAthena, redshift, bigquery?
Answers
Hello, I'm Muntasir Al Qawasmi
I would like to say, in general terms, that the use of what was mentioned, Google and Amazon are quick and innovative solutions, and if used, it is for backup copies.
But in the case of a large company and a large number of daily data, I recommend creating a special program using Python that reads the data and delivers as appropriate for your business policy
I have been helping clients design, build, and deploy data platforms for many years. Initially on premise, but now in the cloud. I think your choice of technology is less important than your chosen data architecture. I would focus on defining this data architecture first (i.e. to satisfy your requirements) and then looking at how the Azure, AWS, or GCP clouds could best satisfy your requirement. I would consider things like data ingestion, orchestration, modelling, and exploitation. The biggest challenges won't be technology, but thinkings like 1) how do you model the data to answer your questions and 2) how do you relate/join the various data sources. I hope this helps; feel free to pop in a call of you would like to delve into more detail or if you have any questions.
Best Choice will be BigQuery + Looker Studio (Data Studio)
Why?
• Super easy to connect to Google Ads, Facebook, Google Analytics, Sheets, etc.
• Natively works with Looker Studio (formerly Data Studio) for reporting.
• Can ingest files from AWS S3, Google Sheets, or APIs.
• Serverless — no infra management.
• Scales well, pricing is based on what you query.
Related Questions
-
How can I aggregate data from online sources about a specific topic?
There are so many ways to do it... Do you need this data for yourself, or you are planning to make a product around it? From what I see you can use Twitter API and Facebook Graph API (Are you comfortable programming?) Most of the students are active on social media so you will find lots of data. Facebook graph API will give you a number of likes and comments to all the posts of you competitors. You can analyze all the posts of your competitors. Using Twitter API you can get all the twits that use certain hashtags or mentions. If you are not into coding, but still want to get social media information, you can take a look at tools like IBM Watson ANalytics ($30 for personal use), it natively connects to Twitter API, and you don't have to be a programmer at all. It is intuitive and easy to learn. Analytics Canvas connects to Facebook Graph API (it's free for 30 days of trial). Unfortunately, you would not be able to collect any personal information from social media at large scale (age, income, gender, etc.), because it violates all the laws about privacy on the Internet. You can use census data instead. Google Sheets are a very handy tool if you are planning to use this information for personal research. You can set up a spreadsheet and add some Java script to make it collect all information from competitor's blogs, and also sites like Reddit. Finally, you can try web scraping (it's not the best, but can speed up the process). A tool like OutWitHub will collect information from websites (such as website reviews) based on the structure you provide (select html tags). You can collect thousands of reviews in one day if you automate it (paid version). Very easy to use. Note: not all the websites are open to this method, review their policies to make sure you are not violating their terms of service. Reviews belong to the website where they were published. If you REALLY need personal data (like how much they earn and how much they spend, etc.), just print out 100 questionnaires and go to Student Union Building of Dalhousie University. Most of the students will share any personal data in exchange for a Tim Horton's gift card that gets them a free coffee. It is probably the least technical and fastest way to get all the data you need. Hope this helps.OT
-
What's the best way for a non-technical person to create advanced forms easily?
Use Google Forms. It works well to conduct surveys. They have lot of the functionality built in. You can use it for a number of other things as well.TP
-
How can I assess the value of my data assets?
I have been helping clients design, build, and deploy data platforms for many years. Initially on premise, but now in the cloud. This is a really good question. My immediate thought is, yes, there is value in this data. The more difficult one to answer is where and how? The obviously answer is monetisation i.e. what would people or organisations be willing to pay for this data. More importantly, is this my data (vs. customer data) to sell? The data may also represent your intellectual property or be your "secret sauce" so you might not necessarily want to sell it. It is then worth asking questions like "what would the impact on my organisation be if I did not have this data?" or "what would the impact on my organisation be if my competitors had this data?". Hopefully this has given you some ideas. Feel free to book in a call if you have any questions or would like to delve into the detail.GR
-
How can I know when to go with my gut or when to trust the data when making a business decision?
In my experience coaching hundreds of people, anytime you go against your gut (aka intuition) you will lose. Data is important to consider, but your gut often knows the path.RC
-
What is the easiest way to store log-in and customer information obtained from our company website?
Well you should store it in a database, most probably an SQL based database. MySQL is a fine start, free and easy to set up. But, you had best start thinking about security issues, especially if you are planning on storing credit card and other personal information. Hackers go right through the standard fare and many companies do nothing to encrypt their data at all, which in some countries is a criminal offense. Best hire a professional to look at your system setup and see what can be done.SK
the startups.com platform
Copyright © 2025 Startups.com. All rights reserved.