I am looking for a basic website crawler to be developed. This must be a standalone application which:
1. Takes in up to 50 Domains
2. Crawls every page
3. Outputs each URL
4. Provides the type of each page found (e.g., HTML, Css, etc)
This is the basic version 1 of the system and there will be future projects to upgrade this.
19 Freelancer bieten im Durchschnitt $774 für diesen Job
Hello, I can get you a Java code or an Excel file that will read all the domains specified in a file and get back the result in a report format of y our choice. Let me know if you are interested