I have a Drupal 7 site that is creating a seemingly infinite stream of sub-directories creating a vast amount of duplicate content. For example:  example.com/test/ - is a legitimate page.  example.com/test/test/ - is a duplicate of the above, and this goes on and on like so:  example.com/test/test/test/test/test/test/test/test/test/test/test/test - etc etc.  All of these pages are crawlable and returning a 200 code. I'm not very familiar with Drupal and I can't pinpoint what is causing this. Any ideas?  Many thanks.
38232
This is in connection to drupal.org issue .  That's when I happen to request for page "node/1/aaa/bbbb", I'm served the response for "node/1" (it's parent page).  I can understand, this is the way Drupal menu system has been designed and works for some obvious reasons.   But I'm in a need to change this. Reason we have some aliased urls across the site, links from third party sites are taking (visitors and crawlers) to not existing page for which ideally http 404 response code and page not found to be returned. Instead Drupal showing it's parent page (with http res. code 200). This is leading to duplicate page in Google webmasters and misleading our genuine users.  I have found a  to this issue. However it is working fine only for 80% of cases, some valid urls (admin/*, sites/default/files/ * , system/ajax/*, etc.,) leads to 404 page.  Would like to hear how others are dealing with this duplicate content issue concerning SEO and possible fix!
103942
Some questions tend to get asked in thousands and thousands of permutations. One such type of question is questions on Apache's mod_rewrite. This   catches most (not all) of them. There are three &quot;please write my rewrite rule for me&quot; type questions today alone. My feeling is that one such question (&quot;I want this and that kind of URL to be redirected to this and that file&quot;) comes up every day at least once. Other examples are:  &quot;Headers already sent&quot; (PHP)  How do I let a container wrap around floated child elements? (HTML)  I get garbled special characters on my web page (HTML / (my)SQL / Unicode)  And hundreds, if not thousands more.   They are always very, very specific, so they can't really be closed as duplicates of anything else. Still, essentially, one quick run-down with a few examples would do for many of these. Instead, they usually get answered after some time. Because of their specificity, and their vast numbers, it is my opinion that many of these questions have little value for future generations. What do you think? Is this being perceived as a problem? Does this warrant a whole new kind of close reason with a link to a FAQ page or reference question or something? That's what floats in my mind as the right solution. Do we need a master question for these to close the other questions as a duplicate of? Bounty starting a bounty to incite some more discussion.
53557
Google clearly states that duplicate content within a single, or multiple, domains is not advised. This is understood, but I am not sure of any exceptions for sites with region-specific content that is often replicated across locales.  For example, a site's /en-us/about page could be identical to /en-uk/about, whereas most likely /en-ja/about is unique.  Are GYM smart enough to understand that the initial URL depth is a locale specifier? Is there any robots.txt or header, etc, trickery that I should include to outline the site's international structure?
43257
You can make a catch-all rewrite rule for apache that acts somewhat like an alias, but can be implemented in a .htaccess file while an alias can't.  As an example, assuming your document root is a hierarchy like this:  /.htaccess /test/ /test/.htaccess /test/file   You can "Alias" all requests to the test folder by making the root .htaccess contain:  RewriteEngine On RewriteRule ^(.*)$ /test/$1 [L]   And the second .htaccess contain:  RewriteEngine On   This works great, and usually is all you need. It just has one problem - it creates multiple valid urls. Now you can access the file with both /file and /test/file.  Ordinarily this isn't a problem, but if you add a file called /test/test you'll see the problem, as the fake alias stops working entirely. If the subfolder has a complex .htaccess of its own it can cause major headaches.  Is there any way to fake an alias from a .htaccess without the original urls still working? (Any RewriteCond for the subfolder that would check if we've already been rewritten or not?)
3474
Possible Duplicate:          Across different sites on different second-level domains exists a universal navigation bar with a collection of roughly 30 links. This universal bar is exactly the same for every page on each domain. The bar's HTML, CSS and JavaScript are all stored in a subfolder for each domain and the HTML is embedded upon serving the page and is not being injected on the client side. None of the links use any rel directives and are as vanilla as can be.  My question is about Google's duplicate content rule. Would something like this be considered duplicate content? Matt Cutt's  about duplicate content mentions boilerplate repetition, but then he mentions lengthy legalese. Since the text in this universal bar is brief and uses common terms, I wonder if this same rule applies.   If this is considered duplicate content, what would be a good way to correct the problem?
41769
In Google Search Console/Webmaster Tools, I found 404 error URLs with this pattern: http://example.com/page-name/example.com  This is happening to all pages, posts, and even category type.  For example:   Here is the .htaccess file:    Does anyone know what might be causing this?
83457
My Google Webmaster Tools account, shows that I have crawling errors that are caused by links to pages that do not exist (404 not found).   But when I look at the source pages that are accused of having these broken links, there are no such links in them.  As of now, I have 6 404 errors in sitemaps and 28 in the rest of my site.  E.g. the address  does not exist, yet Google indicates that I have 3 pages that link to this address:  http://www.webdesk.co.il/sitemapgen.php http://www.webdesk.co.il/articles.php?t=css http://www.webdesk.co.il/articles.php?id=17 //this address dont even exist   If you open each of these pages, you'll see that they don't have these links in them.  I thought this may be a result of an old crawling that was not yet updated, but those pages haven't existed for over than 6 months and Google indicates that all the crawling results are up to date (Dec 22 2012).  Why do I keep getting these 404 errors by Google?
98795
In a project I have very large nginx configuration with a lot of redundant lines.  The current state is like the following  location ~ /loc1/ {     common rules; } location ~ /some/other/location/ {     common rules; } location ~ /yet/anotherone {     common rules; }  etc .....   What I'd like to have is seperated way where there is a generic rule template and a list of locations.  In other related questions people have used regex to match several locations like so  location ~ (/loc1/|/some/other/location/|/yet/anotherone) {     common rules; }   In my case this would result in a huge line which would be very ugly to maintain.  Is there another cleaner way to do this?  Thank you very much.
2831
of Stack Overflow, all content posted on Stack Exchange sites by their users (i.e. you wonderful people) has been provided to the whole universe under . For my fellow non-lawyers, that license basically means:   Anyone can use any Stack Exchange posts at any time without having to ask for permission Making money off of the copied content is permitted You don't even have to copy stuff from here verbatim; you can just use it as a starting point and make whatever edits you want There are just two rules you have to follow:   You have to provide . Simple links to the original post and author info are just fine. You have to  and allow other people to use your content, as long as they follow these very same rules. How meta!    (If you ever forget any of that, and want to refresh your memory, the license info is linked to in the footer of every page.)  There are, in fact, a lot of people who republish varying amounts of our content for assorted reasons. Unfortunately, there are some Stack Content Republishers Attributing Poorly and/or Excelling at Ranking (SCRAPERs, for short).  In this context, "attributing poorly" means any use that doesn't follow our attribution rules or make any other reasonable attempt at give credit. This can get pretty egregious; I've seen SCRAPERs that not only don't link back to SE originals, but also use fake author info and post dates to make it harder to find originals. By "excelling at ranking," I'm referring to copycat sites that end up higher in Google results than the original SE sites do for the same content. That's not necessarily wrong, but in some cases, it indicates inappropriate SEO hackery.  So, the question is: what can you do if you spot a SCRAPER?
12919
I regularly find sites that copy content from elsewhere (mailing lists, etc) while not providing any added value. Quite to the contrary, they are pretty heaped with ads.  I'd like to remove those sites as permanently as possible from my Google search results. Any ideas?
109810
I'm working with a client who manages real-estate agents. All of these agents have their own individual sites, and they all want the same layout and functionality. So, I have a "master" site designed which is dynamically changed to fit the agent/area/whatever through a MySQL database. We have relatively static content in the DB about the agent and the area, then the rest is very dynamic content from a listing service, which is updated daily.  My question is how to set this up so that we can keep on using all the different domains and identical programming (i.e. all the files are identical, except pictures) and have decent SEO as well.  They had some trouble in the past with Google blacklisting their sites because they were considered "mirrored sites". Which...they pretty much are, but the difference between mirrored sites is that we're searching very different areas of the listings DB.  Ideally, all the listings are relatively unique to each site, but sometimes there aren't any listings for the area, or the listings for two sites overlap...that's when they get busted for the mirrorring.  Does anyone have any suggestions on how to both have decent SEO for the individual sites, and keep it easy to manage? There are currently over 100 sites (20 agents, about 5 sites each--one agent site, and a few "area" sites), so not mirroring things is just a ton of work.
66861
I want to replace all my website files for new ones on the server and to test if they work alright but I don't want to be crawled while this is happening or if I am,  I don't want the crawler to be able to see what's in the new page.  I just want to change it for like 15min but i tested other stuff once for 5min and google for webmasters told me they had a lot of 404's during that period and I don't want that to happen again.   Besides that I don't want to have to write my whole .htaccess file. I considered redirecting all the urls to my main page and apply the changes but I think this isn't probably a good practice, any good ideas?
64318
How do I copy an entire directory into a directory of the same name without replacing the content in the destination directory? (instead, I would like to add to the contents of the destination folder)
62002
I have something odd.  I creates some directories using a very simple script :  ... create all directories ( running under a user that has the correct rights ) e.g :  mkdir -p /home/EAI003A/05_EDSNFE/ISU2EAI/VEA/tre   But I get the directories double !  When I do ls -lrt I get   uxel081c: wmb - /home/EAI003A/05_EDSNFE/EAI2ISU/VEA # ls -lrt total 0  drwxr-xr-x    2 wmb      mqbrkrs         256 Jan 28 15:25 tre drwxr-xr-x    2 wmb      mqbrkrs         256 Jan 28 15:25 err drwxr-xr-x    2 wmb      mqbrkrs         256 Jan 28 15:25 arc drwxr-xr-x    2 wmb      mqbrkrs         256 Jun 12 08:35 tre drwxr-xr-x    2 wmb      mqbrkrs         256 Jun 12 08:35 tbt drwxr-xr-x    2 wmb      mqbrkrs         256 Jun 12 08:35 err drwxr-xr-x    2 wmb      mqbrkrs         256 Jun 12 08:35 arc drwxr-xr-x    2 wmb      mqbrkrs         256 Jun 13 10:49 tbt   (notice the double tre, tbt , arc ... )  They double directories do point to the same data.  When you do a normal ls you get   uxel081c: wmb - /home/EAI003A/05_EDSNFE/EAI2ISU/VEA # ls     tre  tre   Thus only two directories !  What happens ?
5695
I recently moved from having a main site like www.example.com with a subdomain associated site yyy.example.com, to moving everything off the subdomain to a subdirectory like www.example.com/yyy/, with the same hierarchy underneath the subdirectory, and now yyy.example.com is just a redirect page to the subdirectory, so yyy.example.com/abc/page-x is redirected to www.example.com/yyy/abc/page-x and so on.  The effect has been that the traffic increased by several times (the total page views are more than ten times at the present rate) what it was to the yyy subdomain. Previously almost all of the traffic to the yyy subdomain was from the www site. This is from monitoring the old yyy site for a year or more.  I'm curious to know if there's any research or results which would indicate whether this is universal (subdirectories beat subdomains) or "just me".
15316
I'm not sure exactly what these "filler pages" are called, so I couldn't do a proper search.  If I have a business, say a plumbing/heating company in NYC, and I created these HTML pages:  New-York-City-plumbing-company.html Manhattan-plumbing-company.html Brooklyn-plumbing-company.html  New-York-City-heating-service.html Manhattan-heating-service.html Brooklyn-heating-service.html   and the content for each page type was the same, except the name of the city/neighborhood changed, i.e.  &lt;h1&gt;[Neighborhood] Plumbing Company&lt;/h1&gt; &lt;p&gt;description of plumbing service&lt;/p&gt;  &lt;h1&gt;[Neighborhood] Heating Service&lt;/h1&gt; &lt;p&gt;description of heating service&lt;/p&gt;   Would this be helpful, harmful, or neutral in terms of SEO? I'm a little afraid Google might see that 95% of the content is the same, minus the name of the neighborhood, and punish my site for it.
32320
how to prevent user creating a duplicate node title for a content type?  I already try uniqueness module () but it's only list out the duplicate content title but not return any error message when user save the new content. Any suggestion module or code? :)