Lunarpages Web Hosting Forum

Author Topic: How to create xml sitemap for more than 50000 webpages? Plz help  (Read 2978 times)

Offline brandhypeabhishek

  • Champion
  • Newbie
  • *
  • Posts: 3
Hello Everyone, How are you, I Hope you all are doing well, please help me guiding about How to create xml sitemap for more than 50000 webpages? Any help will be highly appreciated. Thanks in advance.

Offline MrPhil

  • Senior Moderator
  • Berserker Poster
  • *****
  • Posts: 6425
Re: How to create xml sitemap for more than 50000 webpages? Plz help
« Reply #1 on: November 24, 2017, 07:47:22 AM »
You have more than 50k unique pages that you want indexed? I presume that you didn't create them by hand, but are running a blog, store, or something like that, to generate them. Anything of that nature will have sitemap-creation add-ons or plug-ins available, to generate the XML file upon demand. Some will do the whole site at once, and would be run on cron job every week or even every day, and others may update the XML file any time a page is added, changed, or deleted.

You should read Google's documentation and see if you will have any penalties for that many pages. If there are, you may want to think about only sitemapping certain critical pages, such as category head pages (and everything above them), ignoring lower level pages (at least, those older than a certain age). This could bring your sitemap down to a manageable size.
« Last Edit: November 24, 2017, 09:46:58 AM by MrPhil »
Visit My Site

E-mail Me
-= From the ashes shall rise a sooty tern =-

Offline brandhypeabhishek

  • Champion
  • Newbie
  • *
  • Posts: 3
Re: How to create xml sitemap for more than 50000 webpages? Plz help
« Reply #2 on: November 27, 2017, 02:29:53 AM »
Thank You MrPhil. I am using screaming frog to generate the sitemap. FYI the site is providing online homework help and have multiple questions in it.

Offline MrPhil

  • Senior Moderator
  • Berserker Poster
  • *****
  • Posts: 6425
Re: How to create xml sitemap for more than 50000 webpages? Plz help
« Reply #3 on: January 27, 2018, 03:24:00 PM »
Update: It appears that Google limits you to 50,000 entries in a single sitemap (which was apparently the original question). AFAIK, you can provide multiple sitemap files, and a master sitemap_index.xml file to point to all of them.
Visit My Site

E-mail Me
-= From the ashes shall rise a sooty tern =-

Offline Woo Woo

  • Newbie
  • *
  • Posts: 2
Re: How to create xml sitemap for more than 50000 webpages? Plz help
« Reply #4 on: May 07, 2018, 01:51:28 AM »
If your sitemap includes images (or videos or something else), you can create separate sitemap and add it to the robots.txt. Or you can just create separate sitemap for categories and product pages.
It's better not to have one sitemap with 50000+ pages. It's official google's recommendation.
Also you sholud remember that sitemap have to contain only important pages (not pagination, filter or sorted pages). You should review what pages you need to add to the sitemap.

Offline saisaanvi

  • Newbie
  • *
  • Posts: 4
Re: How to create xml sitemap for more than 50000 webpages? Plz help
« Reply #5 on: May 03, 2019, 06:04:36 AM »
it is really a tough job and some tools can provide site map upto some extent. but for 50,000 pages  you have to write manually