• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

DAVID.MU

  • Experiments
    • Indexing
    • Crawling
  • About me
  • Contact
  • Cambiar a ESCambiar a ES
  • You are here: Home / Experiments

    Experiments

    Does Google follow redirects to pages blocked by robots.txt?

    David Leave a Comment

    TL;DR: No, at least in this experiment Google has evaluated the final page of the redirects and when it was blocked in robots.txt it has not followed the redirect. The goal of this experiment is to verify how the Google crawler reacts when it finds a redirection that points to a URL blocked with robots.txt […]

    Filed under: Crawling

    Does Google index the words present in a domain name?

    David Leave a Comment

    thompson

    TL;DR: Yes, Google has indexed the words present in the domains used for this experiment and includes them in the information retrieval process, showing the domains as results to a search containing the word. The goal of this experiment is to check if Google takes into account the words present in a domain when selecting […]

    Filed under: Indexing

    How to find out when a website was published?

    David Leave a Comment

    TL;DR: In order to find out the approximate date when a website was published you have to: 1. Search the URL in Google. 2. Add &as_qdr=y15 at the end of the URL showing up in your browser bar after searching and then hit enter. 3. The date will appear to the left of the search […]

    Filed under: Indexing

    Primary Sidebar

    CATEGORIES

    • Crawling (1)
    • Indexing (2)

    LATEST POSTS

    • Does Google follow redirects to pages blocked by robots.txt?
    • Does Google index the words present in a domain name?
    • How to find out when a website was published?
    linkedin/in/davidmunozsanchez/

    DAVID.MU LLC

    1342 NM-333, STE C #1486 1071, Tijeras, New Mexico 87059, USA

    • Cookie Policy
    • Privacy Policy
    • Terms of service
    You accept cookies from this site? more info