I am looking for a basic website crawler to be developed. This must be a standalone application which:
1. Takes in up to 50 Domains
2. Crawls every page
3. Outputs each URL
4. Provides the type of each page found (e.g., HTML, Css, etc)
This is the basic version 1 of the system and there will be future projects to upgrade this.
20 Freelancer bietetn im Durchschnitt $763 für diesen Job an
Hello, I can get you a Java code or an Excel file that will read all the domains specified in a file and get back the result in a report format of y our choice. Let me know if you are interested