Web Scraping

Step-by-Step Instructions

1. Navigate to Knowledge Map.

2. Select Files

3. Select Add Web Content.

4. In the pop-up window:

   • Enter the URL of the webpage you want to upload.

   • Select the relevant Tags to categorize the content.

5. How much content should we capture:

   1. This page only: Captures just the URL you entered.

   2. Deep Crawl: Follows multiple levels of links. Useful for documentation sites or blog categories. May take longer and capture many pages.

      1. If you want, you can select the option to Set up weekly auto-sync. This will automatically re-crawl the URL every week and keep your knowledge base up to date with any changes made to the page.

6. Hit Submit.

What Happens Next?

1. Iris Processes the Content:

   • The system scrapes the provided URL and extracts the page content.

   • Iris converts the extracted information into an MD file (Markdown format) for better compatibility and readability.

   • A timestamp is automatically added to track the upload date.

2. Title Generation:

   • The title of the document is dynamically generated based on the webpage's metadata or content structure.

Manage An Existing Web-Sync

EdIris will regularly crawl your support sites and add new articles, update existing articles, and delete articles that have been removed

1. Go to Settings > Sync Rules (or click here)

2. Select the Web Sync Tab in the top right corner

3. Here you can:

   1. Add a new Web Sync

      

      1. Select + Add Site

      2. Ensure New URL is selected

      3. Enter URL, pick Tags, and select Add to Sync

   2. Edit an existing Web Sync

      1. Click the trash can icon to stop Iris from auto-syncing those websites

      2. Click on the name of the site URL to open that link in a new tab

Refreshing a Web-Sync

Want to replace a webpage with a more current version?

Simply select Actions > Refresh next to any uploaded webpage.