Robots.txt file is the first file read by bots and crawlers to determine whether to index and read a particular page/ link. This process helps…