The List Website Entries module enables effortless extraction of data from any website with listings. Whether you’re looking to compile all the houses from a real estate site or gather every product from an online marketplace, this node allows you to easily capture and manipulate that data. You can use this module to provide live contextual information from different sources to an AI model. This module has multiple configurations that can enhance its capabilities.
  • Extract as list: If enabled, outputs the search results as a list instead of a single value.
  • Extract as: Options are Markdown or JSON.
  • Navigate pages: Attempt to navigate pages if the webpage being read has pagination. Picture a website that has product listings and they continue in other pages.
  • Advanced Options:
    • Continue on Error: If enabled, in case the scraping fails, the output value of Text will contain the value <ERROR>, instead of failing the workflow
The List Website Entries module has one input and one output:
  • Input: -Website, provide the URL of the marketplace or listing page. Ensure that the link is already sorted according to your preferred criteria, as this node is designed to read and process pages that list items in the specified format.
  • Output:
    • Search Results, the results of the listing in either markdown or JSON format, as previously selected