Loading...
Answers
MenuScraping information of shopping websites according to price ?
For open APIs this is possible but how to do with the closed ones ?
Answers
You can either dissect the inner workings of ajax calls (if the website has any) or write a regular full-page scraper which traverses the product catalog.
Popular choices for scraping are casperjs, beautiful soup, mechanize, etc. Each having its strengths and drawbacks - casper for instance can emulate the browser behavior, while the rest are much less resource intensive but often require going through extra hoops when the website is complicated.
We have some in house technology WCML that we use to mine data on the web. Let me know if we can be of assistance.
Web Scraping has become a pretty common practice on the Internet. There are many different web scraping toolkits now and you can generally get product and price information, as long as it is part of the HTML. In general, you would not do a lot of decisioning at the scrape level but would gather all information in a database and sort from there.
The challenge is that web scraping is dependent on how the web page is structured in the HTML. As the store's pages change over time, you will need to address your scraping.
Your competitor selection should start from the closest, then you can move to a broader market. For that, you can start by creating a list of potential e-commerce platforms that can prove to be your competition. You can include as many competitors as you think are suitable. All the visible competitors are on your list. The whole process of research will help you create a huge list of competitors. Most of the time, e-commerce sites present a huge amount of data to crunch. The complicated part is to extract this data. This way, you can manage the whole price extraction without any hassle. The amount of data is going to be huge. You can go with a third-party storage system to keep your data safe and secure. This will allow you to access the extracted data from anywhere anytime. Following all the above-mentioned steps, you will conveniently gather relevant and valuable price data without spending much time.
You can read more here: https://www.promptcloud.com/blog/retailers-guide-to-price-scraping/
Besides if you do have any questions give me a call: https://clarity.fm/joy-brotonath
Related Questions
-
How should the dynamic between a ux designer and a developer who are working together look like?
It depends a lot of in the skill sets and experience of both people but in most cases the ux designer should be controlling the developer pretty heavily in order to make sure his ideas come through properly. The UX designer may just need to work on his approach so people don't feel bossed around and more like they are working together. In an ideal world, there would be a project manager who makes sure everything is communicated well and keeps the dynamic feeling great.JM
-
Which is the best hosted free bug tracking tool for a team size of up to 10 members?
Being a freelance developer for the last 10 years or so, I have seen and used almost every project management software you can think of, open-source and closed, and I have found that the "best bug tracking tool" completely depends on your process. I employ and direct teams in an Agile process that involves loose story-based requirements with point-based estimates, two-week cycles of iterative development, planning and retrospective sessions, and look-ahead and show-and-tell meetings with the stakeholders. It's important to me, then, that my tools have a method of capturing all those pieces with as much detail as I need but no more. On projects, I've successfully used Pivotal Tracker, Unfuddle, Redmine, Codebase HQ, Basecamp, Trello, and many others. I usually recommend people to Trello for light, agile management; it's essentially a digitized version of sticky notes and swim lanes. If your team actually needs a full-featured ticketing system, see Redmine (Rails), Trac (Python), or Mantis (PHP), depending on your language preference. Hosting a version of these yourself is fairly trivial, and numerous options exist for cheap or free hosted versions. For something in the middle, Github Issues is a good fit and the open-source clone GitLab.org replicates most of those features nicely. However, if you're looking for a hosted version, you're probably looking to offload that tricky "backup" thingy, and in that case, how important is your data? How proprietary? What's your business model? If you're working on an open-source project, Github will give your team a free account with private repos, issues, wikis, and the like. If you're okay with your project being "readable by all", public projects on Pivotal are still free. I'm mostly a consultant these days, leading small groups of junior or intermediate developers into a more productive, more mature, fully operational teams of senior software developers. One of the first things I teach folks is how to use a project management system... and why! It'll save your bacon if it's simple, effective, and reliable. With a few minutes of discussion about your project, I can probably help you select the right tool and service for your team. Let me know if I can help with that. Best of luck!DR
-
How do I run a successful closed beta for my web application that is almost done with development?
Create an ideal customer profile. Create some questions that will allow to you survey a potential tester to determine if they fit your profile. Design simple landing page with very clear value proposition that speaks to your ideal customer. Ask for a minimum of information up-front (email), but ask for more info after they commit by submitting the first piece of info. (KISSmetrics does a good job of this on their current website trial signup). Use the their answers to these profiling questions to put the applicants into buckets. Let in the most ideal bucket first, or split them into groups if they're big enough. Try and measure engagement the best you can. Measure qualitative and quantitative data. Schedule calls with your beta testers to find out more, especially with the ones who's user behavior seemed to indicate that they didn't get value from your product. Find out why. Make sure they are indeed your ideal customer. Pick up the phone and get to know your customers inside and out. Meet them in person if possible. Incorporate their feedback quickly and get more feedback. Rinse repeat.DH
-
What's the best way to build a MVP web app that handles order management, purchasing, invoicing, supplier management and inventory?
The best way to build an MVP for any SaaS product is to create a landing page that looks like a real product. Here's an example of one I built. http://www.happiily.com In this case, it advertises the primary features of the product and invites people to sign-up. When they do, they are asked for information which qualifies the person and then sends me an email. I built this quickly and very inexpensively and started getting inbound leads from it shortly thereafter. I got on the phone with each person who signed-up and explained the features I wanted to build and was able to do a lot of customer learning based on that. Happy to talk to you in a call if you'd like to talk more about customer development with SaaS products.TW
-
How can a small offshore development company find companies/software sales people to sell their service in the US/UK?
My company does a lot of consulting with offshore firms who are looking for a way to generate new business, so I hear this question a lot. My first reaction is that you need to totally reverse your mindset when you talk about your own company. You mentioned that you have: a great software developers team, proven track record, passion, real value But, everyone says that. There a 10,000 companies that have those things, so a customer isn't going to notice it. You need to figure out what your company is best at (doesn't have to be technical) and present it as a solution to a specific problem that clients have. Maybe a speciality, or really good project management, really good communications, a special expertise or experience, a personality, experience with a certain type of client.. really anything.. But, there must be some thing that makes your company 'special' otherwise you will be lost in the mix. Don't worry about things like rates, or the fact that you have 'great' developers. Those are generic. Think about why a client would really choose you, and try to build on that! After you understand your company identity, it gets much easier to identify and engage marketing channels because you understand your target.DH
the startups.com platform
Copyright © 2025 Startups.com. All rights reserved.