How do data providers (FB, Acxiom, 4square, etc) license out their data to enterprise with ideas around pricing?

I have worked on projects handling transfer of large image files across multiple vendors and being served on web for different targets (mobile, web etc.). I have also implemented cloud and DevOps solutions for high volume/scale of data and transactions. While I could tell you a list of technologies which can be used - the problem here is more of design and architecture. So I will start with that. Also a lot of decisions are based on more information and the use case - so I will need information on certain aspects of your application. 1) For searching an image users will enter search terms. You will have to associate the search terms/tags to actual images. While there are multiple ways to do this association - what is the link between a search term and image in your case. Do you have a separate database/service which will link given search terms to given set of images? 2) Since you are talking about handling millions of images - they will need to be partitioned logically or bucketized for optimum way to store and search. Is there a natural distribution? Do your users access all the images all over the world? 3) For any data storage related to images - we will need to go one detail further. While relational DB might serve a lot of needs, there might be use cases where considering other storages will be needed. Some questions which will lead to answers are - what kind of data per image will be stored? How frequently this data might change? How many new additions will happen on this archive? How fast would you want the updates to be visible to users/consumers? 4) What kind of traffic do you expect roughly? This will decide a lot design around auto scaling, load balancing and other aspects. 5) I am not a front end expert - but similar principles of scaling would apply to front end stack. Though I can not necessarily answer what technologies might fit your needs best.

Qualifications on the answer: Created the first website to be ever commercially licensed by a ministry of education for in-school use (Brainium 1996). Was a VC associate at a 500,000,000 firm called Knowledge Universe focused on all things education-related that intersected with technology. Short Answer: Yes and no. Longer Answer: Learning analytics is one of those "lightening in a bottle" kind of industries. Everyone knows it's going to happen, whomever makes it happen at scale first, will have a huge advantage, and even more niche plays in this space will still create massive value for shareholders and significant transformation of education as we know it. That's the good news. The problem is that funding startups in the education space that are dependent on "permission" being granted by existing institutions and their employees is very much hit or miss. It's getting easier to raise a $500-800k seed round for a good idea with a good team, but the problem is that getting the next bigger round is very challenging in that the models of most of these seed-funded companies are not growing fast enough to prove to institutional investors that a growth model of the idea being very big in a relatively short time-period is credible. So if you're intent on pursuing this, I'd be sure to focus entirely on how you can get growth and adoption with requiring little to no institutional buy-in which sounds almost impossible on the surface of the area you're exploring but I believe isn't actually impossible. I'm very passionate about this space and want something like what you're describing to succeed so would be happy to do a quick call to hear where you're at and see if I can provide you some actionable ideas on how to reduce the friction / improve adoption of your analytics product.

I have been helping clients design, build, and deploy data platforms for many years. Initially on premise, but now in the cloud. This is a really good question. My immediate thought is, yes, there is value in this data. The more difficult one to answer is where and how? The obviously answer is monetisation i.e. what would people or organisations be willing to pay for this data. More importantly, is this my data (vs. customer data) to sell? The data may also represent your intellectual property or be your "secret sauce" so you might not necessarily want to sell it. It is then worth asking questions like "what would the impact on my organisation be if I did not have this data?" or "what would the impact on my organisation be if my competitors had this data?". Hopefully this has given you some ideas. Feel free to book in a call if you have any questions or would like to delve into the detail.

In my experience coaching hundreds of people, anytime you go against your gut (aka intuition) you will lose. Data is important to consider, but your gut often knows the path.

Separate, but related note: "NYC Open Data" is growing every day & provides free data https://opendata.cityofnewyork.us/ For national data check out the parent organization Socrata.com. Great resources for public data records, which are 100% free, contains everything from real estate transactions to the number of public bathrooms... Best of luck tracking down the costs.