Question What is the difference between robots.txt and meta robots? and Which is the best?
Robots.txt is used to restrict entire directory or folders.
Meta robots is used to restrict each pages.
Robots.txt is a file located in your website root file that gives or denies access to certain files.
Meta robots is located in every page header and gives or denies access to certain files.
Adding a URL to robots.txt will prevent the page from being crawled but it can still be indexed.
Adding meta robots "noindex" will prevent a page from being indexed but it can still be crawled.
If you want a page to be accessible to crawling but not indexed (because it links to other interesting parts of your website), then you don't block it in robots.txt and add the appropriate meta robots to make google no index the page but still follow the links.
Case 1: Robots.txt forbids indexing of a URL but meta tag allows it.
Outcome: Page will be blocked by robots.txt, meta tag will never be seen by googlebot thus will be ignored, page will only be shown in results as a reference in very rare occasions (it will not have a description, only title and URL).
Case 2: Robots.txt allows indexing of a URL but meta tags forbids it.
Outcome: Page will not be indexed and will not be shown in the search results at all.
Case 3: Both Robots.txt and meta tag forbid the indexing of a URL.
Outcome: Page will be blocked by robots.txt, meta tag will never be seen by googlebot thus will be ignored, which will allow the page to be shown as snippet with no description in rare occasions in the search results.
Meta noindex vs meta index conflict: If a single page has both meta noindex and meta index tags, meta noindex will resolve. The page will not be shown in the search results.