Loading...
Answers
MenuI am building an image archive that will ultimately have millions of images you can search. I'd love the hear your thoughts on technologies to use.
My list so far:-
Storage - S3
CDN - CloudFront or CloudFlare
Database - ?
Front-End Framework - ?
Answers
I have worked on projects handling transfer of large image files across multiple vendors and being served on web for different targets (mobile, web etc.). I have also implemented cloud and DevOps solutions for high volume/scale of data and transactions.
While I could tell you a list of technologies which can be used - the problem here is more of design and architecture. So I will start with that. Also a lot of decisions are based on more information and the use case - so I will need information on certain aspects of your application.
1) For searching an image users will enter search terms. You will have to associate the search terms/tags to actual images. While there are multiple ways to do this association - what is the link between a search term and image in your case. Do you have a separate database/service which will link given search terms to given set of images?
2) Since you are talking about handling millions of images - they will need to be partitioned logically or bucketized for optimum way to store and search. Is there a natural distribution? Do your users access all the images all over the world?
3) For any data storage related to images - we will need to go one detail further. While relational DB might serve a lot of needs, there might be use cases where considering other storages will be needed. Some questions which will lead to answers are - what kind of data per image will be stored? How frequently this data might change? How many new additions will happen on this archive? How fast would you want the updates to be visible to users/consumers?
4) What kind of traffic do you expect roughly? This will decide a lot design around auto scaling, load balancing and other aspects.
5) I am not a front end expert - but similar principles of scaling would apply to front end stack. Though I can not necessarily answer what technologies might fit your needs best.
Related Questions
-
How do I find the Total Addressable Market for a Big Data product like Datameer?
As someone who has built TAM models for various industries - I can attest this isn;t always easy. Many analyst firms (former analyst here) typically do some surveys and research among public company revenue reports for the year and then do some pixie dust extrapolating. The hard thing about a market where startups like Datameer play is that it is a) a nascent market and b) not a "zero sum" game. This makes it difficult to a) fully understand WHICH companies actually are in the market for this flavor of BI and b) know how the space is developing in terms of white space AND disrupting the entrenched BI giants. You might want to look at the press published versions (read: free) of analyst market share reports, and extrapolate from their. Also, back-channel gossip may give you some revenue numbers on these players - and if you assume pipeline is 3-4x revenue, build a model from there. Hope this helps.MS
-
How can I know when to go with my gut or when to trust the data when making a business decision?
In my experience coaching hundreds of people, anytime you go against your gut (aka intuition) you will lose. Data is important to consider, but your gut often knows the path.RC
-
Programming language for web platform?
NOT Wordpress! Facebook uses PHP among other languages, if that tells you anything. Every programmer will be bias toward what they like and what they are used to coding with. The bigger question for you is, have you completed your due diligence? Is there a real need for another crowd funding platform? Why would yours be so different that it would stand out among the others? Who is you target market? What is your unique selling proposition? How big is your potential market? Should you go vertical or horizontal? You may have already answered these and the two dozen other questions every entrepreneur must ask before they waste valuable time and money to become another business start up statistic. Assuming you have a strong grasp of where you are heading and a solid strategic blueprint to be able to predict your growth, the last thing I will leave you with is this. The programing language is not nearly as important as your database structure and IT infrastructure for the purpose of scale. Not to say the language is not important - all three components work together. Spend time on the other two and work backwards. If you would like to discuss this further at no charge, use the link below. I provide a free 30 minute consultation to first time callers. https://clarity.fm/kevinmccarthy/FreeConsult Best regards, Kevin McCarthy www.kevinmccarthy.comKM
-
What is the best programming language for building multi-platform mobile software that is scalable?
I've been involved in several projects that hinged upon this question (generally start-ups or web+mobile apps), and it's not a clean or easy answer unfortunately! Plus every developer you talk to is going to try to sell you their services, but what you need is simply the truth! I'd suggest that you consider shifting your focus away from finding a best language. That sort of premise can eliminate options that are actually quite valid solutions. Keep in mind that any mobile app will require multiple languages working in harmony - all while used in a commercially consistent and standards-based manner. That's the part that determines your technical scalability! Almost every programming language can achieve this functionality with an experienced developer on your team. The best advice I can give you, is to consult with at least two people on this, and three if your app is os-specific. The most important insight will come from a "full stack" developer. One who's got a variety of enterprise experience, and can code at all levels of the stack. This person needs to have experience in leading a team of other developers, which forces the strategy of which technologies to use and why onto their plates daily. Secondly, you'll want to consult with a "front end" developer, who can tell you what's possible using advanced OOP JavaScript techniques (like Google's angular.js), because in an absurdly fast change over the past 3 years, much of the formerly back-end work has made it's way to the front-end, and is driven by JavaScript, predominantly JSON data, and awesome API's. The game has changed and the front-enders are the poor souls dealing with this rapid shift daily. To do this, they're also fantastic JavaScript programmers, which is a language that runs on all mobile devices and all browsers, too. Love it or hate it JavaScript is the most commonly used language in the world. Finally, if your app is OS-specific, you'll want to consult with a developer who works predominantly with the OS your app is built for. This person lets you know what's possible from the device POV, should know what stacks and JavaScript approaches can and can't be done on that OS, and how to leverage the resources of the device for your app as well as extend it's functionality. BONUS - loop back to the full stack developer to double-check the claims of the front-ender and the mobile developers. Always double check with your most senior programmer ;) All of these consults together in addition to your own research and due diligence will get you comfortable and allow you to navigate on this rather daunting but deeply important journey. There aren't any turn-key options. Instead it's a series of inter-connected modules driven by different languages, and all working in tandem. Every solution will have bugs, and no one group of technologies can do everything without proper developers. If you'd like to go further down the rabbit hole, then we should definitely set-up some time to speak. Otherwise, I wish you great luck in research and encourage you to learn as much as you can! :) It's going to seem hard, and might give you a headache here or there, but learn everything you can about how different technologies "talk" to each other, and then you will be able to build a map for keeping your app and business scalable regardless of the changing tides of technology!MM
-
What is the best way to do a permissions system for a low traffic administration system, PHP preferred, but general suggestion is also valuable.
Hi, I have been a PHP developer for 13 years and have experience building enterprise applications. What framework are you using? Most PHP frameworks have a built in authentication and authorization mechanism. For example, Yii has a builtin RBAC system that allow you to define a hierarchical permission system. If you're not using a framework or you just want to implement your own system for whatever reason then you can utilize Object Oriented Design and implement a Base controller (if MVC) that checks the permission for each request. That won't use a lot of if statements. Probably need more info, to give definitive guidance. Hope that helps.AP
the startups.com platform
Copyright © 2025 Startups.com. All rights reserved.