This time I will give a tutorial about how to update a robots.txt that we have on blogger before that, let's get to know a little about robots.txt
Robots.txt is a rule that is usually created automatically by bloggers, which is a product of Google to make it easier for other products from Google to access our website and use the features of these products such as Google Adsense, Analytics, Console etc.
And one of the important features of robots.txt is telling the "crawler" google search engine which url or page that contains elements that we don't want to share such as images, videos, audio. etc. which can be set using this robots.txt For more details, you can read the article via the link below.
how to setting custom robots txt for blogger
Prevent article images from appearing on Google search using robots.txt
Alright that's a little bit about robots.txt, ok then let's start updating robots.txt if you want to know how to add rules to robots.txt you can read it via the link above
first
We start from getting the robots.txt file, if you already have it you can go directly to the second step. There are 2 ways to get the robots.txt file, the first way is to copy it directly on our website, with our website url then we add /robots.txt or like the image below
After you finish copying it then you just need to paste it in a text editor application like notepad and replace it as you want. Remember, if you save the file name, the robots format must be in .txt(Box1) also saved using UTF-8(Box2) encoding. like the picture below
Another way is to use the robot.txt test from Google via the link below
After you are redirected to the site, then you look for the button, please select property and you select your website so that the image looks like below
Next, you click the send button (Box1) then download (Box2) the robot.txt file will be downloaded soon
Second
Well above is how to get the robot.txt file if you want to edit it on a platform or server other than blogger, to update robots.txt on blogger there are 2 ways the first way is to use the tool in the url that I first gave in the first step that is testing robots.txt you can access it via the link below.
After choosing the properties or blogs that we have, then you only need to edit the rules in the robots.txt to be clearer what we can edit you can read it in this article #Link how to edit robots.txt as shown below
After the editing process is complete, we save or update it by clicking the send button (Box1) and update (Box2)
after that we wait for the robot.txt to be changed after google crawls our website again you can check it in the latest version(Box1) when was the last time google crawled our website and if there has been a change there(Box1) as shown below
The method above has a weakness that we cannot control the crawling process from Google and will take a lot of time, the next method I will give is
Using custom robots.txt, using this method will immediately update our robot.txt on the website even though in the robots.txt testing tool there will be no changes because changes only occur if Google crawls our website.
The trick is that we go to the blogger website and then click the settings button (Box1) then we look for the indexing and crawl section (Box2) then we activate custom robots.txt(Box3) as shown below
Next, we click the custom robot.txt (Box4) button and a display like the one below will appear, we fill in the robots.txt file rules that we have edited and then save
Now your robots.txt has changed you can see it by using the above method that adds /robots.txt in your blog url
Alright, here's the tutorial this time, hopefully it's useful, thank you for visiting, if you have any questions, please comment below, thank you.
0 Komentar