Loading...
Answers
MenuScraping information of shopping websites according to price ?
For open APIs this is possible but how to do with the closed ones ?
Answers
You can either dissect the inner workings of ajax calls (if the website has any) or write a regular full-page scraper which traverses the product catalog.
Popular choices for scraping are casperjs, beautiful soup, mechanize, etc. Each having its strengths and drawbacks - casper for instance can emulate the browser behavior, while the rest are much less resource intensive but often require going through extra hoops when the website is complicated.
We have some in house technology WCML that we use to mine data on the web. Let me know if we can be of assistance.
Web Scraping has become a pretty common practice on the Internet. There are many different web scraping toolkits now and you can generally get product and price information, as long as it is part of the HTML. In general, you would not do a lot of decisioning at the scrape level but would gather all information in a database and sort from there.
The challenge is that web scraping is dependent on how the web page is structured in the HTML. As the store's pages change over time, you will need to address your scraping.
Your competitor selection should start from the closest, then you can move to a broader market. For that, you can start by creating a list of potential e-commerce platforms that can prove to be your competition. You can include as many competitors as you think are suitable. All the visible competitors are on your list. The whole process of research will help you create a huge list of competitors. Most of the time, e-commerce sites present a huge amount of data to crunch. The complicated part is to extract this data. This way, you can manage the whole price extraction without any hassle. The amount of data is going to be huge. You can go with a third-party storage system to keep your data safe and secure. This will allow you to access the extracted data from anywhere anytime. Following all the above-mentioned steps, you will conveniently gather relevant and valuable price data without spending much time.
You can read more here: https://www.promptcloud.com/blog/retailers-guide-to-price-scraping/
Besides if you do have any questions give me a call: https://clarity.fm/joy-brotonath
Related Questions
-
How can I take an idea for a new app and turn it into a real product?
Here are some options: ___Free Options___ 1) Make apps without needing to spend a lot of time learning to code. Look into "MIT App Inventor" (http://appinventor.mit.edu/explore/). It's a very _easy_ way to make pretty complex apps using drag and drop coding. You can find youtube tutorials that will show you how to make your first app within 5 minutes. I've used MIT App Inventor to make prototype apps for many of my ideas, saving me tens of thousands of dollars if I paid a dev to do it instead. Learn to do "real" coding yourself. The main investment will be your time. There are plenty of free resources for learning coding on the web. I'd suggest learning "React Native", it's a relatively new way to code apps, which allows you to make one app that will run on both Android and iOS. 2) Find a software engineer cofounder. Go to Meetups, conferences, local hackerspaces / makerspaces. Hang out on relevant online forums (e.g. https://www.reddit.com/r/startups/ make sure to read their rules for posting before posting though). It won't be easy to find a tech cofounder, your idea will need to be amazing, and you yourself will need to be very passionate and capable in order to convince someone to partner up with you for sweat equity. ___Paid options___ 1) Use developers with less established reputations / portfolios (lower cost, higher risk) Be very wary of freelancers on sites like Upwork, Fiverr, etc. Here are some basic hiring rules: In your hiring script, make sure to ask for all applicants to give their account name for github/bitbucket and Trello. Don't hire agencies, only hire individuals. To get hired, ask them to do a simple task via Trello and submit the code via github/bitbucket. This task should only take them maybe 1 hour. Check the quality of what the applicants and if they deliver it in a timely manner. Keep the 1 or 2 people that do a good job. If you don't do this vetting these "low cost" developers may end up costing a lot in the longer run. 2) Use developers with more established reputations / portfolios (higher cost, lower risk) With this option your app will probably cost > $20k to develop, but it can be worth it if you have a single idea that you know you want to move forward with. I can introduce you to a very high quality developer in NY if you're at this stage. Let me know if you'd like any additional help more tailored to your specific app ideas, best, LeeLV
-
Which is the best hosted free bug tracking tool for a team size of up to 10 members?
Being a freelance developer for the last 10 years or so, I have seen and used almost every project management software you can think of, open-source and closed, and I have found that the "best bug tracking tool" completely depends on your process. I employ and direct teams in an Agile process that involves loose story-based requirements with point-based estimates, two-week cycles of iterative development, planning and retrospective sessions, and look-ahead and show-and-tell meetings with the stakeholders. It's important to me, then, that my tools have a method of capturing all those pieces with as much detail as I need but no more. On projects, I've successfully used Pivotal Tracker, Unfuddle, Redmine, Codebase HQ, Basecamp, Trello, and many others. I usually recommend people to Trello for light, agile management; it's essentially a digitized version of sticky notes and swim lanes. If your team actually needs a full-featured ticketing system, see Redmine (Rails), Trac (Python), or Mantis (PHP), depending on your language preference. Hosting a version of these yourself is fairly trivial, and numerous options exist for cheap or free hosted versions. For something in the middle, Github Issues is a good fit and the open-source clone GitLab.org replicates most of those features nicely. However, if you're looking for a hosted version, you're probably looking to offload that tricky "backup" thingy, and in that case, how important is your data? How proprietary? What's your business model? If you're working on an open-source project, Github will give your team a free account with private repos, issues, wikis, and the like. If you're okay with your project being "readable by all", public projects on Pivotal are still free. I'm mostly a consultant these days, leading small groups of junior or intermediate developers into a more productive, more mature, fully operational teams of senior software developers. One of the first things I teach folks is how to use a project management system... and why! It'll save your bacon if it's simple, effective, and reliable. With a few minutes of discussion about your project, I can probably help you select the right tool and service for your team. Let me know if I can help with that. Best of luck!DR
-
I have a great app idea, and I need help bringing it to life.
I'm not sure if this is how you imagine this world to work, but at least according to the order you wrote it "raising funds" was first. In reality it should actually be one of your final steps of the stage you are at right now. It may even come after a year or two! So you have this great app idea, and you're looking for a place to start... Don't! Don't start yet before you decide whether you have what it takes to get into a roller coaster that can ruin your life and make you miserable! Not trying to scare you but I think most people only hear about these great success stories. They have this dream of maybe, possibly, becoming the next big thing... Because they have the best idea for an app... You don't hear about the failures so often. And even if you do, you don't hear about what the founders of these failing startups had to go through. Truth is you are most likely gonna fail. And I'm saying that without even knowing what your idea is. There are so many barriers on your way that even a great product with a great team is likely to fail. Some people would say "I'm not afraid of failing", "It's good to fail cause you learn", "Failing will make me stronger for the next startup". That's somewhat true but it doesn't mean that failing is easy. As oppose to what people sometimes say - you do not want to fail! It's very painful!!! You have to understand what failing in a startup means. You can work your a$s for 2-3 years, have little to no salary, waste other people's money (most likely your friends and family first), lose friends, fight with your partners, your family, your spouse, devote 20 hours a day for your startup all this time, forget about the little and big things you used to enjoy in life, and only then, after debating 100 times whether you should quit or not, you finally decide that it's not gonna work and you've failed. Disappointing your family, your investors, yourself. Trust me it is painful. Are you sure you wanna do this to yourself? If yes, give me a call. I have the experience you need! From idea stage, to proof of concept, to running beta tests, getting millions of millions of users in ways you can't even imagine, creating features and experience that will make these millions of users completely addicted and viral, raise money in a smart way, hire the right people, find a great co-founder, succeed, fail, be persistent, and enjoy the ride! Good luck, RoyRM
-
What would be a good approach for marketing a software development businesses?
For software development business LinkedIn ads, content marketing and Google Adwords don't work well. The best and most cost effective method is email outreach. Try to find the contacts of key decision makers in Bay Area from your target companies. You want to present yourselves as custom mobile and web development specialists and highlight your core competencies to get an initial call to discuss their mobile strategy or software development needs. Attaching your case studies how you helped other similar businesses and your portfolio can be extremely helpful as well. Try to always focus on the benefits in you pitch that they can get by working with you and point their missed opportunities of not having certain types of software or apps for their business. Clients love that software development companies have not only strong execution but also ideation skills. Hope this helps. If you have any questions I am also available on call for your convenience.AA
-
How can a small offshore development company find companies/software sales people to sell their service in the US/UK?
My company does a lot of consulting with offshore firms who are looking for a way to generate new business, so I hear this question a lot. My first reaction is that you need to totally reverse your mindset when you talk about your own company. You mentioned that you have: a great software developers team, proven track record, passion, real value But, everyone says that. There a 10,000 companies that have those things, so a customer isn't going to notice it. You need to figure out what your company is best at (doesn't have to be technical) and present it as a solution to a specific problem that clients have. Maybe a speciality, or really good project management, really good communications, a special expertise or experience, a personality, experience with a certain type of client.. really anything.. But, there must be some thing that makes your company 'special' otherwise you will be lost in the mix. Don't worry about things like rates, or the fact that you have 'great' developers. Those are generic. Think about why a client would really choose you, and try to build on that! After you understand your company identity, it gets much easier to identify and engage marketing channels because you understand your target.DH
the startups.com platform
Copyright © 2025 Startups.com. All rights reserved.