Loading...
Answers
MenuScraping information of shopping websites according to price ?
For open APIs this is possible but how to do with the closed ones ?
Answers
You can either dissect the inner workings of ajax calls (if the website has any) or write a regular full-page scraper which traverses the product catalog.
Popular choices for scraping are casperjs, beautiful soup, mechanize, etc. Each having its strengths and drawbacks - casper for instance can emulate the browser behavior, while the rest are much less resource intensive but often require going through extra hoops when the website is complicated.
We have some in house technology WCML that we use to mine data on the web. Let me know if we can be of assistance.
Web Scraping has become a pretty common practice on the Internet. There are many different web scraping toolkits now and you can generally get product and price information, as long as it is part of the HTML. In general, you would not do a lot of decisioning at the scrape level but would gather all information in a database and sort from there.
The challenge is that web scraping is dependent on how the web page is structured in the HTML. As the store's pages change over time, you will need to address your scraping.
Your competitor selection should start from the closest, then you can move to a broader market. For that, you can start by creating a list of potential e-commerce platforms that can prove to be your competition. You can include as many competitors as you think are suitable. All the visible competitors are on your list. The whole process of research will help you create a huge list of competitors. Most of the time, e-commerce sites present a huge amount of data to crunch. The complicated part is to extract this data. This way, you can manage the whole price extraction without any hassle. The amount of data is going to be huge. You can go with a third-party storage system to keep your data safe and secure. This will allow you to access the extracted data from anywhere anytime. Following all the above-mentioned steps, you will conveniently gather relevant and valuable price data without spending much time.
You can read more here: https://www.promptcloud.com/blog/retailers-guide-to-price-scraping/
Besides if you do have any questions give me a call: https://clarity.fm/joy-brotonath
Related Questions
-
How do I run a closed beta test for my mobile application? Development will be finished in 3 weeks.
You should try to engage people using social networks, it is easier to spread than email. The conversion rate on emails are low but is still a valid tool for that. Send and email with a simple and objective message that will make people want to try. The best way to have feedback from users is to watch them use the app. You should put them on the hands of everybody that you can and without any instrucions and just watch, don´t even say that the app is yours. Try to do it a lot. If you want feedback from others, you can include the feedback form inside the app and suggest users to answer occasionally. I would also strongly recommend to use a tool as Flurry Analytics. Is the best way to get data from how is the use of the application. Pay attention to those data and be open to change your app a lot, you may need more features or cut some off to make it easier to use. If you need more help please contact me.BS
-
How should the dynamic between a ux designer and a developer who are working together look like?
It depends a lot of in the skill sets and experience of both people but in most cases the ux designer should be controlling the developer pretty heavily in order to make sure his ideas come through properly. The UX designer may just need to work on his approach so people don't feel bossed around and more like they are working together. In an ideal world, there would be a project manager who makes sure everything is communicated well and keeps the dynamic feeling great.JM
-
Can my experience with building "no code" applications be translated into something that will impress hard core coders?
Your skills would be most useful if you were starting your own startup and needed to make an early prototype to show to investors or potential cofounder developers. Your experience in debugging, testing, and agile, could help you get a job as a product manager, and the fact that you have a background in some sort of 'coding' will help too. It's very unlikely that it would help you get an actual dev job though, since you wouldn't be able to translate your programs into actual code that could be taken over/continued by other devs. Even if the programs you mentioned do allow you to export as code, it's unlikely that it would be exported in a way that's very usable by other devs.LV
-
What would be a good approach for marketing a software development businesses?
For software development business LinkedIn ads, content marketing and Google Adwords don't work well. The best and most cost effective method is email outreach. Try to find the contacts of key decision makers in Bay Area from your target companies. You want to present yourselves as custom mobile and web development specialists and highlight your core competencies to get an initial call to discuss their mobile strategy or software development needs. Attaching your case studies how you helped other similar businesses and your portfolio can be extremely helpful as well. Try to always focus on the benefits in you pitch that they can get by working with you and point their missed opportunities of not having certain types of software or apps for their business. Clients love that software development companies have not only strong execution but also ideation skills. Hope this helps. If you have any questions I am also available on call for your convenience.AA
-
How do I run a successful closed beta for my web application that is almost done with development?
Create an ideal customer profile. Create some questions that will allow to you survey a potential tester to determine if they fit your profile. Design simple landing page with very clear value proposition that speaks to your ideal customer. Ask for a minimum of information up-front (email), but ask for more info after they commit by submitting the first piece of info. (KISSmetrics does a good job of this on their current website trial signup). Use the their answers to these profiling questions to put the applicants into buckets. Let in the most ideal bucket first, or split them into groups if they're big enough. Try and measure engagement the best you can. Measure qualitative and quantitative data. Schedule calls with your beta testers to find out more, especially with the ones who's user behavior seemed to indicate that they didn't get value from your product. Find out why. Make sure they are indeed your ideal customer. Pick up the phone and get to know your customers inside and out. Meet them in person if possible. Incorporate their feedback quickly and get more feedback. Rinse repeat.DH
the startups.com platform
Copyright © 2025 Startups.com. All rights reserved.