Loading...
Answers
MenuScraping information of shopping websites according to price ?
For open APIs this is possible but how to do with the closed ones ?
Answers
You can either dissect the inner workings of ajax calls (if the website has any) or write a regular full-page scraper which traverses the product catalog.
Popular choices for scraping are casperjs, beautiful soup, mechanize, etc. Each having its strengths and drawbacks - casper for instance can emulate the browser behavior, while the rest are much less resource intensive but often require going through extra hoops when the website is complicated.
We have some in house technology WCML that we use to mine data on the web. Let me know if we can be of assistance.
Web Scraping has become a pretty common practice on the Internet. There are many different web scraping toolkits now and you can generally get product and price information, as long as it is part of the HTML. In general, you would not do a lot of decisioning at the scrape level but would gather all information in a database and sort from there.
The challenge is that web scraping is dependent on how the web page is structured in the HTML. As the store's pages change over time, you will need to address your scraping.
Your competitor selection should start from the closest, then you can move to a broader market. For that, you can start by creating a list of potential e-commerce platforms that can prove to be your competition. You can include as many competitors as you think are suitable. All the visible competitors are on your list. The whole process of research will help you create a huge list of competitors. Most of the time, e-commerce sites present a huge amount of data to crunch. The complicated part is to extract this data. This way, you can manage the whole price extraction without any hassle. The amount of data is going to be huge. You can go with a third-party storage system to keep your data safe and secure. This will allow you to access the extracted data from anywhere anytime. Following all the above-mentioned steps, you will conveniently gather relevant and valuable price data without spending much time.
You can read more here: https://www.promptcloud.com/blog/retailers-guide-to-price-scraping/
Besides if you do have any questions give me a call: https://clarity.fm/joy-brotonath
Related Questions
-
What would be a good approach for marketing a software development businesses?
For software development business LinkedIn ads, content marketing and Google Adwords don't work well. The best and most cost effective method is email outreach. Try to find the contacts of key decision makers in Bay Area from your target companies. You want to present yourselves as custom mobile and web development specialists and highlight your core competencies to get an initial call to discuss their mobile strategy or software development needs. Attaching your case studies how you helped other similar businesses and your portfolio can be extremely helpful as well. Try to always focus on the benefits in you pitch that they can get by working with you and point their missed opportunities of not having certain types of software or apps for their business. Clients love that software development companies have not only strong execution but also ideation skills. Hope this helps. If you have any questions I am also available on call for your convenience.AA
-
Can my experience with building "no code" applications be translated into something that will impress hard core coders?
Your skills would be most useful if you were starting your own startup and needed to make an early prototype to show to investors or potential cofounder developers. Your experience in debugging, testing, and agile, could help you get a job as a product manager, and the fact that you have a background in some sort of 'coding' will help too. It's very unlikely that it would help you get an actual dev job though, since you wouldn't be able to translate your programs into actual code that could be taken over/continued by other devs. Even if the programs you mentioned do allow you to export as code, it's unlikely that it would be exported in a way that's very usable by other devs.LV
-
How can a small offshore development company find companies/software sales people to sell their service in the US/UK?
My company does a lot of consulting with offshore firms who are looking for a way to generate new business, so I hear this question a lot. My first reaction is that you need to totally reverse your mindset when you talk about your own company. You mentioned that you have: a great software developers team, proven track record, passion, real value But, everyone says that. There a 10,000 companies that have those things, so a customer isn't going to notice it. You need to figure out what your company is best at (doesn't have to be technical) and present it as a solution to a specific problem that clients have. Maybe a speciality, or really good project management, really good communications, a special expertise or experience, a personality, experience with a certain type of client.. really anything.. But, there must be some thing that makes your company 'special' otherwise you will be lost in the mix. Don't worry about things like rates, or the fact that you have 'great' developers. Those are generic. Think about why a client would really choose you, and try to build on that! After you understand your company identity, it gets much easier to identify and engage marketing channels because you understand your target.DH
-
How do I run a closed beta test for my mobile application? Development will be finished in 3 weeks.
You should try to engage people using social networks, it is easier to spread than email. The conversion rate on emails are low but is still a valid tool for that. Send and email with a simple and objective message that will make people want to try. The best way to have feedback from users is to watch them use the app. You should put them on the hands of everybody that you can and without any instrucions and just watch, don´t even say that the app is yours. Try to do it a lot. If you want feedback from others, you can include the feedback form inside the app and suggest users to answer occasionally. I would also strongly recommend to use a tool as Flurry Analytics. Is the best way to get data from how is the use of the application. Pay attention to those data and be open to change your app a lot, you may need more features or cut some off to make it easier to use. If you need more help please contact me.BS
-
How can I take an idea for a new app and turn it into a real product?
Here are some options: ___Free Options___ 1) Make apps without needing to spend a lot of time learning to code. Look into "MIT App Inventor" (http://appinventor.mit.edu/explore/). It's a very _easy_ way to make pretty complex apps using drag and drop coding. You can find youtube tutorials that will show you how to make your first app within 5 minutes. I've used MIT App Inventor to make prototype apps for many of my ideas, saving me tens of thousands of dollars if I paid a dev to do it instead. Learn to do "real" coding yourself. The main investment will be your time. There are plenty of free resources for learning coding on the web. I'd suggest learning "React Native", it's a relatively new way to code apps, which allows you to make one app that will run on both Android and iOS. 2) Find a software engineer cofounder. Go to Meetups, conferences, local hackerspaces / makerspaces. Hang out on relevant online forums (e.g. https://www.reddit.com/r/startups/ make sure to read their rules for posting before posting though). It won't be easy to find a tech cofounder, your idea will need to be amazing, and you yourself will need to be very passionate and capable in order to convince someone to partner up with you for sweat equity. ___Paid options___ 1) Use developers with less established reputations / portfolios (lower cost, higher risk) Be very wary of freelancers on sites like Upwork, Fiverr, etc. Here are some basic hiring rules: In your hiring script, make sure to ask for all applicants to give their account name for github/bitbucket and Trello. Don't hire agencies, only hire individuals. To get hired, ask them to do a simple task via Trello and submit the code via github/bitbucket. This task should only take them maybe 1 hour. Check the quality of what the applicants and if they deliver it in a timely manner. Keep the 1 or 2 people that do a good job. If you don't do this vetting these "low cost" developers may end up costing a lot in the longer run. 2) Use developers with more established reputations / portfolios (higher cost, lower risk) With this option your app will probably cost > $20k to develop, but it can be worth it if you have a single idea that you know you want to move forward with. I can introduce you to a very high quality developer in NY if you're at this stage. Let me know if you'd like any additional help more tailored to your specific app ideas, best, LeeLV
the startups.com platform
Copyright © 2025 Startups.com. All rights reserved.