Google Penguin Update | Return On News

Posted on May 8, 2012 by tpltx70

Google has been very busy in the past month or so. First they rolled out an update to how they handle anchor text, during March. Then we had another Panda tweak. And finally, the big daddy “Penguin” update hit in late April.

Google Penguin Information

For those of you who don’t know about Penguin yet, it has caused quite a stir. Google originally dubbed it the “Over Optimization” Penalty. That threw much of the SEO world into a panic, so they retitled it the WEBSPAM update (a.k.a. Penguin). The thought is that they’d systematically drop sites that exhibit spammy behavior from the index.

What defines spammy behavior in this case? Some of the types of behaviors I’ve seen mentioned include:

Creating a link farm, or link circle
Having a link profile stacked with questionable backlinks to the website
Selecting a keyword-rich domain and launching just shallow content
Publishing low quality content that is clearly machine generated
Keyword-stuffing on the page or entire domain

Hopefully your website survived all these changes unscathed. If so congratulations, but don’t just sit on your laurels and enjoy it. You need to know what behaviors are out of bounds, so it’s crucial to read all you can about Penguin. There’s something called negative SEO that has been the buzz. That’s something you don’t want aimed at you.

Google Penguin Resources

Penguins, Pandas, and Panic at the Zoo (SEOmoz)

Of course, I have to start with SEOmoz. They are my favorite source for interpreting pretty much anything Google says about SEO. Start with this article first, and peruse the blog listing for other related posts. They are all valuable.

The following are some of the more interesting and insightful posts from other sites that I’ve found covering Penguin. If you see anything else worth adding, let me know and I can update the post.

Original Webmaster Blog Post: Another step to reward high-quality sites

Here is the official announcement of Penguin straight from Google. They did a lot of advanced commentary, but this was when we knew it was set in motion.

Google Penguin Update: Don’t Forget About Duplicate Content (WebProNews)

This is both a refresher on duplicate content and commentary on managing it in the age of Penguin. There are a bunch of educational videos embedded at the end, so consume this one when you have time to watch all the Matt Cutts cameos.

Google’s Unnatural Links Messages: The Shot Heard ’round the SEO World? (Search Engine Watch)

This post does a great job of explaining what the new rules of backlinks are. This is a different kind of algorithm update. It’s the first one I can recall where they can penalize your website for the lack of quality of those who choose to link to it. The days of volume backlink campaigns is over, my friend. Quality is now a defining factor.

Google Crushes Spinners and Spammers (Website Magazine)

Website Magazine chimes in on everything with at least a high level overview. This does that and hits on a few areas where spammy behavior might be identified. Quick read if you don’t mind a more high level view of the situation.

Webspam And Panda Updates: Does SEO Still Matter? (WebProNews)

First, this post is not advocating the belief that SEO is done, dead, or otherwise obsolete. It does cover some of how it will need to change moving forward, and also includes a list of sites that were impacted, either positively or negatively, from April 24 forward.

What to Do if Your Search Rankings Were Hurt by Google’s Penguin Update (Hubspot)

Hubspot does a good job of explaining how you can figure out if you’ve been penalized by Penguin. It also provides some basic tips to take into account if you do get hit. Most importantly, it links to the form you can submit to Google for their review if you think you’ve been penalized in error. Just be sure you’ve fixed whatever they told you to fix in Webmaster Tools before you submit, and your chances of success will higher.

Move Over Panda, a Penguin’s on the Loose (Site Reference)

This one has some good tips about how to address any concerns you have over Penguin. It also outlines some areas to start with if you are in remediation mode.

If My Content is “King,” Why Does Google Hate It? (Site Reference)

Google just wants everyone to remove the shortcuts and play by the same rules. This talks about some realities around content marketing. It also touches on how the white hat SEO practitioner can insulate against further algorithm jockeying.

Google Penguin Update: Impact of Anchor Text Diversity & Link Relevancy (Search Engine Watch)

This ties to the March announcement, which was a precursor to Penguin. They’re cracking down on shady offpage practices, and you need to alter your backlink efforts accordingly. Exact match anchor text is no longer the way to go.

Even the FTC is getting in on the fun

Have you heard about the FTC antitrust investigation surrounding this update? That is probably the only way we’ll ever see Google even consider rolling back an update. Although it’s a very long shot that this will be successful at proving any guilt, it’s worth watching. The last time we had a major investigation like this it was Microsoft, and that was quite a ride!

Thanks for reading!

Anchor Text, Trackbacks, What to Tweet, and More | Return On News

Posted on March 28, 2012 by tpltx70

What a month March has been. Google pre-announced some massive changes to come to backlinks for their algorithm, SXSW has come and gone, and I’ve had quite a few distractions on the personal front.

I plan to do a whole summary of the upcoming Google algorithm changes later, but for now, let’s see what else has caught my eye this month.

Are Over Optimised Link Profiles A Barrier To Top 5 Rankings?

SEO Wizz has authored a number of posts on backlink campaigns and how to handle anchor text. The standard direction from most SEOs is that you should acquire as many backlinks that use anchor text which is an exact match to your most important keywords.

This article not only questions that long-standing belief. It shows data to support the right way to do it. I’ve found some great content on this site, and most of the time it’s pretty spot on.

5 Things You Need to Know About Trackbacks

If you use WordPress or similar blogging platforms, surely you’ve seen trackbacks in the user interface. Ever wonder why those matter? It doesn’t just show who links to you or who you link to. There’s much more, and it even helps your site authority with Google. For anyone generating blog content, this is a must read even if just for informational purposes.

7 Facts Extroverts Should Know about Introverts in Business

Being an introvert doesn’t mean you’re just socially inept or shy. It’s a whole different way of generating internal energy and thinking. If the above is a must read for bloggers, this is a must read for everyone.

If you’re an extrovert, it helps you understand why everyone else doesn’t get their energy from interactions with you and others. If you’re an introvert, it affirms that there’s nothing wrong with you for being different in a society that rewards extroverted behavior. Seriously, take a moment to read and understand this and it will only help, no matter whom you are.

Extreme Homepage Makeover: How to Increase Your Conversion Rate 106%

Hubspot reviews how to massively improve conversion rate from the home page, with a nice list of the key variables they manipulated to make it happen. Good stuff.

17 Foolish Mistakes to Avoid as a Guest Blogger

Guest blogging is one of the best ways to build your name and reputation, network, and get backlink love for your own blog or website. But you can really mess it up if you don’t stop to think about your approach first. This covers the main areas that tend to happen with overall strategy, pitching your guest posts, and writing the content itself.

Google Analytics Social Reports Coming Soon

Google keeps beefing up the data available in some areas, just as they are removing some visibility to keyword-level data. They’re clearly pushing social over the traditional SEO practices. This gives you an idea of where their thought process is heading in the early going.

What Marketers Should and Shouldn’t Tweet

Hubspot shares some insightful research that advises the right ways to use Twitter, and what to avoid. If you’ve been struggling to get followers and build a network on there, this might help you understand why.

Link Karma: How Linking to Others Can Get You Lots of Links in Return

You hear the themes about “pay it forward” in social media all the time. Here’s one way to use social networking for search engine optimization link love. I, personally, found this to be a great post.

Summary

That’s it for March folks. If you see anything I missed this month that was ground breaking for you, share below so all can read it too!

Why SEO is Similar to Fantasy Baseball

Posted on February 10, 2012 by tpltx70

Two of my biggest interests these days are search engine optimization and fantasy sports. You might be surprised to hear the responses I get when others hear that they are both so interesting to me. Seriously, I get everything from “What a geek!” to “Why the heck would you like either of those?”

If you are one of the people who doesn’t “get” either of the topics, that is perfectly fine. Let me share my rationale for why the two have parallels that seem to work well with the way I think and operate

Reasons Why Search Engine Optimization is Similar to Fantasy Baseball

Data Intensive

Both SEO and Fantasy Baseball are extremely dependent on statistics and data analysis. They both require trending, deep dives into why things are happening, and deriving conclusions about what will happen in different possible future scenarios. Essentially, they both involve studying statistics, and studying an even deeper level of statistics than most people will both with. In baseball, there’s Sabermetrics and other similar analytical systems. In SEO, we have relevance, competition, and the like.

I’ll take your SEO competition score and Page Rank, and raise you a BABIP and a couple of FIPs.

Balances Analytical with Creative Thinking

Balance between analysis and creativity is another area where both disciplines (and yes, both are disciplines in their own way) mirror each other.In SEO, we find high leverage keywords with low competition, then gear our targeting and messaging around it. In Fantasy Baseball, we identify those players whose stats do not truly support their level of performance, and manage our team in a way that exchanges overperformers for underperformers. Essentially, it’s using numbers to come up with creative ways to extract value where no one else sees it yet.

Changes Daily

The rapid rate of change is possibly the biggest parallel between the two topic areas. In any sport, injuries are simply part of the game. Rotations / starting gigs change, players retire or get suspended, and in many cases players are added from the minor leagues or elsewhere mid season (or demoted back down). In business, the competitive landscape can change not only daily, but hourly! In both cases, the changes ripple through in many ways, and it’s up to you to figure out how to respond in the best way possible.

Social Media and Content Heavy

SEO and Fantasy Baseball are both very visible topics around the social media properties. Both are generating content on a daily basis and looking to share that content, then exchange opinions and ideas. There are fanatics in both camps, folks who will spend every waking hour thinking about new ways to win the game. Social is the best way to spread content and talk about it, and both groups have figured that out in short order.

Deep Level of Expertise Required

There are many times where someone finds out I’m into either or both of these topics, and my standard response is, “I’m a geek. I love this stuff.” And sadly, that’s 100% true! I’ve spent years reading every piece of analysis I could find, trying to work out how to apply it, and even generating my own creative ideas for handling the different challenges that crop up. To get really good at either practice, you need to have a depth of knowledge that is unrivaled by generalist types. And you have to absolutely LOVE collecting that information and using it.

Summary

There you have it, my take on why SEO and Fantasy Baseball aren’t really different. I brainstormed at least a dozen different angles on this spin, and the above are all that made the final cut. Are you into both topics? What commonalities do you see?

Anchored Links: What You Need To Know for SEO

Posted on January 16, 2012 by tpltx70

I was posed an interesting question a few weeks ago, and have been doing a bit of testing and research against it. The question was related to anchored links, and whether web crawlers would index only parts of a web page if routed there via an anchored link.

Anchored Links vs. Anchor Text

Before diving into this topic in detail, I want to point out the distinction between anchored links and anchor text.

Any good SEO can tell you what anchor text is – the actual words you use when hyperlinking to another website or internal web page. For a long time, anchor text has been known to impact SEO by matching backlinks to keywords overtly.

Anchored links are completely different. Did you ever click a hyperlink, only to watch the page load and immediately jump to the most relevant spot on the page? If you did, you clicked an anchored link.

Anchored Links: How They Work

You can always tell you are looking at an anchored link if you see a URL with a pound sign (#) and a word after it. For example (sample only, non-working link):

http://returnonnow.com/Sample-page#Anchor

This URL tells the browser to first load the page, and then move it so a pre-specified location moves to the top or main real estate of the viewing area, depending on how far down the page is. How does it know where to anchor?

Before assembling the above URL, you will need to define what the anchor point on the page is. This is done using the “name” tag. To do so, assign a value to “name” that is the same as the “Anchor” in the above URL.

Example Anchor Code:
<a name=”Anchor”>Sample Anchor Point</a>

In this example, the anchor will point at the location on the page where the phrase “Sample Anchor Point” is placed. This is a rather basic HTML command that even the most inexperienced of webmasters (even marketing folks with very limited HTML knowledge) can employ.

Anchored Links and SEO: Answering the Question

Anchored links are most useful for enhancing user experience. By loading the most relevant content on the page right upon click, you provide the reader with immediate access to the content they are seeking. The fewer clicks and less confusion you have site visitors go through, the better.

As for SEO, anchored links do have an impact. The impact is not on the web crawler as the aforementioned question suggested. Anything after the “#” is considered a browser-only command, and is completely ignored by spiders (probably to prevent partial indexing of pages). The browser merely uses it to route to the content for UX reasons.

Anchored links, however, do provide two useful SEO benefits:

The keyword used as the anchor is noted by the web spider much like anchor text is. It is a much lesser indicator of relevance, but taken into account nonetheless.
For longer or more complex pages, search engines frequently create direct links to the anchored locations. These are positioned very similarly to sitelinks in function. You can see a sample of anchor links showing as sitelinks below. Click on each of them and see where it takes you for reference. I’ve hyperlinked to the SERP directly if you click the image.

: Click the image to test drive the anchored links for yourself

Summary

That should give you plenty to think about for anchored links. Surely there are some nice tips and tricks that others have identified. If you have any great ideas that I overlooked, feel free to share in the comments below. All ideas area welcomed!

3 Keys to Balancing User Experience and SEO

Posted on December 12, 2011 by tpltx70

I cover a variety of SEO topics on Return On Now, but it is often important to take a step back and consider the full realm of online marketing.

While SEO is most important for presenting your best game face to the search engines, it is not the only variable at play. You also have to keep a close eye on usability.

Usability: What is It?

Usability is a very intuitive concept. Essentially, it answers the question of how easy it is for the average web user to navigate around the website?

Usability includes:

How easy or difficult it is to learn what the site is about on first entry
How quickly it is to navigate to the most relevant content on the site (faster nav and fewer clicks is the goal)
Whether buttons, navigation, and other interactive objects are placed where most users will be able to find them without undue frustration

Well managed usability provides the best possible experience for the readers. This is what we refer to a User Experience (UX).

Balancing User Experience with Search Engine Optimization

There are some common misconceptions about this topic among marketing types. The worst one is that you cannot balance UX and SEO without one of the two (or both) suffering in some way. This is simply not true.

The objective of building a site for both readers / users as well as the search engines is both reasonable and achievable. Sure, you may have to make some tradeoffs in how you architect your site or structure your content to accommodate both needs. But the key point is that minor tradeoffs between the two can result in major gains in user satisfaction, without causing undue negative impact to traffic volumes.

Here are the 3 keys to balancing SEO and UX:

Design your site layout, templates, and architecture for the REAL users. If the site provides a stellar user experience, traffic will come back and grow over time via word of mouth, sharing, and other means. It will also grow via SEO, as one of the factors in Google’s algorithm measures the overall usability of the site.
Structure the content – the title, headers, body content, and image alt-tags – in the best way possible for the search engines. I’m not saying to write a bunch of keyword-stuffed gobblety-goop either. Write to communicate clearly to the average reader, but also be sure you are speaking in the language that people use to search.
Craft your overall site content strategy to provide high value, regularly updated material. Particularly with a newfound focus on timing via the Google Fresh update (October, 2011), you cannot build a site and never touch it again. If you want to move up in the rankings, you need to offer relevant, timely, and shareable materials. Did I mention that you need to write for the readers and not the search engines?

Summary

You may hear from various sources that it is difficult or even impossible to balance SEO and user experience / usability. Those sources are simply misinformed. If you follow the guidelines above, you should be able to deliver on both goals.

Have you found it challenging to balance the two? Let me know what your biggest frustrations are below and I’ll see if I can help.

Subdomains vs. Subfolders: Which is best for SEO?

Posted on November 21, 2011 by tpltx70

There have been some long standing myths about what ranks best for search engine optimization between subdomains and subfolders on your root domain. Both approaches can have value for SEO purposes. However, in my experience, the vast majority of marketing and web practitioners have an incomplete or inaccurate understanding of when and how to use them.

Let’s take a moment to look at this topic in more detail.

Definitions

Subdomain

While your root domain will show up as domain-name.com or www.domain-name.com, a subdomain shows up as subdomain.domain-name.com. Good examples of subdomains are:

As you can see, a subdomain is most freqently used when a specific area of the website is targeted to a very pointed topic or audience. AdWords is focused on Google’s PPC offering, the second link is the support site for a company called TigerTech, and the last example is the community website for my current employer, SolarWinds.

Subfolder

A subfolder is essentially a directory on the overall hierarchy / structure of your root domain itself. In your browser address bar, this shows up as www.domain-name.com/subfolder/. If you click on any of the keywords in my navigation, you can see what subfolder each of those tag items sits in.

Subdomain vs. Subfolder: When to Use Each

Let’s start with subfolders, since it is very straightforward to understand when to use them and how they work.

Subfolders are integral components on your root domain. When in doubt about where to put new content, ask yourself how important that content will be to getting your root domain ranked.

If it is a crucial piece, put it in a subfolder. It will serve to help increase your keyword coverage, grow your site page volume, and position you for relevant backlinks deeper than your home page (which we all should be chasing).

Subdomains can be a tougher challenge. For several years now, Google has given direction that subdomains are not considered part of the root domain. However, earlier this year, they advised that some changes have been made to how they handle subdomains.

This post led to quite a bit of confusion. Many practitioners, and even SEO experts automatically read this post to say that subdomains are now being considered as part of the root domain. Those individuals are dead wrong.

If you read it carefully, this is changing nothing about subdomain vs root domain ranking except for one thing – links between various subdomains and a root domain are now considered “internal links”.

This is a great modification in reality. Previously, gray hat SEOs could simply buy one domain for $7 – $20, and roll out a slew of subdomains on similar topics. Then, they could create a link farm by cross linking all of the various properties. That was a great way to cheat by hyper-optimizing the on-page and then providing relevant backlinks across the sites.

Now, each subdomain counts as a separate website for ranking purposes, but links between two subdomains are counted as internal.

No more games, no more link farms. Take that Gray Hat!

How to Use Subdomains for SEO

So now that we’re on the same page, you can see why the subfolder is the obvious answer for generating content and receiving direct benefits from it. But why would any reasonably cognizant SEO recommend using a subdomain? I’m glad you (I) asked!

In today’s web-heavy world, a new field called online reputation management (ORM) has emerged. A large piece of this practice is managing the SERPs for your name and/or brand terms (e.g. company or product names).

Without going into to much detail on ORM (which actually deserves a full post of its own), here’s the key point: subdomains can rank independently for a keyword. Google often hides multiple pages on one domain that rank for the same keyword and forces you to click “view more results” to see them. They only do this for same-domain URLs.

Many people worry that having multiple sites trying to rank for one keyword is too much work, and that very well may be true. But how much is it worth to own the two of the top 5 SERP positions for your most important terms or brand names? The only two ways to do this are to roll out a subdomain or launch a new website on a whole new root domain. At least the latter gives you external links, if you don’t mind managing a split brand (not recommended).

Summary: They are Not the Same for SEO Purposes

There is a long standing confusion that subdomains are considered part of your root domain, like subfolders are. This is simply not true. Subdomains are separate sites that are considered internal to your root domain only for counting backlinks. Subfolders are the best way to drive SEO for the root domain.

What other uses have you found for subdomains? Any creative ideas I missed here?

White Hat SEO vs. Black Hat SEO

Posted on November 3, 2011 by tpltx70

White Hat SEO vs. Black Hat SEO: What's the difference?

This week I received an inquiry focused on White Hat vs. Black Hat SEO from a college student in MIS. This inquiry was part of an e-business class assignment he received, in which he was to write a report on what White Hat SEO is.

Having taken the time to respond thoughtfully, I believe it is useful to share my quick-hit answer about what they are and how they impact SEO success. I’d very much like to hear any feedback you or your favorite SEO may have, so please share the post freely and comment at will!

His specific questions:

What do you think the most challenging aspect of white hat seo?
Is SEO really ethical? I mean. white hat/black hat…..It´s all about gaming the search engine to get better placements. Don’t you agree that SEO practices are in constant change as the Internet evolves?
Who is responsible to drawing the line between white hat SEO and black hat SEO?

My answer:

Let me just respond to the three questions in aggregate.

First, there is a huge divide between white hat and black hat SEO techniques. Black Hat involves using any method possible to game the system, not following SEO best practices as outlined by Google, and most often, using shady means to leap ahead of other reputable sites in the SERPS.

The whole Google Panda update was to draw a line between white and black hat (i.e. spammers in most cases) practitioners, and they actually did a decent job of doing so. That said, they overshot their target and ended up hurting some fully ethical, white hat sites. This was mostly collateral damage or by association, since part of Panda is to compare your site, content, etc. with “like” websites on similar topic areas.

White Hat SEO is not rocket science. It is about building reputable websites without an ounce of deception involved. It is about generating high quality, share-able content that readers will want to read and spread to their network. And it is about truly adding value with that content.

Black Hat SEO is easy to identify. It might include keyword stuffed, hard to read material. It might have bad grammar or punctuation. It might thinly mention the topic of what it wants to rank for, sticking the keyword in all the right “on page” areas, but really just thinly veiling a pitch for some spammy/scammy product. Most of all, if you search for a term and get to their site, a reasonably intelligent consumer would quickly want to bounce away from it or even be offended at the bait and switch. And they often acquire backlinks through paid or other mistrusted means (in Google’s Eyes).

The hardest part of White Hat SEO is two-fold:

Being disciplined and focused enough to stick to your guns with standards, and to execute on a well-thought out and constructed content and linking pursuit strategy.
Convincing other stakeholders in your company or organization to avoid using questionable techniques.

#1 is a ton of work and requires a lot of focus. #2 is political in most cases.

Last Words

How would you describe White Hat SEO, or answer these questions if posed to you? I typed this out in a matter of minutes, so it is my natural, quick response. Anything I missed?

Website Traffic – The Power of Buzz

Posted on October 18, 2011 by tpltx70

We talk about search engine optimization (SEO) quite a bit on here, but SEO is (obviously) not the only way to drive web traffic.

You could also invest in various paid placements such as PPC, display, or sponsored white papers to drive targeted traffic back to your website. Many old school marketers immediately turn to these tactics, and the can certainly be used effectively if managed properly.

There’s something even bigger at your disposal.

The Buzz

None of the pay-for-play methods can compare to the potential scale of widespread buzz about your company. It is the pinnacle of word-of-mouth, albeit with wider, louder, and deeper reach. Whether fleeting or lasting, buzz can really make magical things happen for your business.

Great, So What Can We Do To Generate Buzz?

The best way to generate positive buzz is to be outstanding. Not average or good enough, but beyond expectations. Offer products or services that really wow your audience, that deliver on the vision of Guy Kawasaki’s Enchantment. Look at what Apple has done and imagine the possibilities.

Of course, not every company can have an image like Apple. Fortunately, there are a few things you can do to attract positive attention to just about any business.

Public Relations (PR)

PR is available to anyone, individuals, businesses, non-profits, special interest groups, performance artists…you name an industry or profession, and there is a benefit within reach. This can be as simple as getting to know the local business page editor, or as sophisticated as you are comfortable taking your program.

As a powerful awareness vehicle, PR has a positive impact on organic as well as direct traffic to your site.

Social Media

Low on budget but have a lot to say? Passionate about, or possibly a budding thought leader, in your field? Start a blog. Take a few moments to learn about content marketing,

and how you can spread the word via social media. You will meet some amazing people if you jump in with an open mind and a good idea of what you are trying to accomplish.

Charitable Donations

Philanthropy is always a good way to show your commitment to the local community or bigger causes. Donations in the form of money, relief for emergency victims, or even assistance putting on a charity event are all equal in value. And it helps those in need, which is a tremendous side benefit on a personal level.

Publicity Stunts

I have seen some outrageous but madly successful publicity stunts over the years.

Publicity Stunt Gone Bad (Image via Wikipedia)

When they work well, they can leave a lasting memory / impression. But be careful here, because a publicity stunt has been known to backfire for rather unexpected reasons.

As a sub set of publicity stunts, viral video or other materials is also a great “digital publicity stunt” that you can test in a controlled fashion. Just make the video good and cutting edge, and the audience could respond with raving approval.

What are some other creative ways you have used to drive traffic to your website?

Robots.txt: Understanding the Basics of Crawler Management

Posted on September 12, 2011 by tpltx70

Business owners today would be hard pressed to develop ongoing consistent business without depending on web leads and traffic. As part of your overall online marketing arsenal, it is crucial to have the right documents posted live on the web.

One of the most important files to post is robots.txt. Since search engines use “bots” (a.k.a. spiders, wanderers, or crawlers) to index websites and pages, Google has made available a feedback loop.

The feedback loop is essentially the ability to request that search engines stay away from certain pages or directories completely. This is done using the robots.txt file.

Take note that it is a suggestion, so it may or may not prevent crawling over time. There are many reasons why a page might still get crawled (e.g. spider following a backlink right to the page), so it is imperative that you hide any sensitive or proprietary data behind a login or other access layer as well.

Robots.txt: How to make it

This file is quite literally a plain text file that you can create in minutes using MS Notepad or a similar basic application. Simply open up Notepad, create a new file, and save as “robots.txt”. Voila! You have the file.

Robots.txt: Managing the content

So we have the file, but it contains no content just yet. Let’s look at what goes in there.

The robots.txt file is basically divided into several sections, one for each of of the robot crawlers’ User Agent names. You can direct a section at all crawlers or a specific one, so this can be as simple or as complex of an exercise as you feel comfortable taking on.

Each section begins with code designating what User Agent is targeted. Examples of this piece of code include:

User-agent: * (targets all spiders)
User-agent: Googlebot
User-agent: insert name of agent here

Beneath each User-agent designation, there will be one or more DISALLOW entries. How’s that for simple-to-learn logic? The syntax for this command looks like this:

Disallow: / (tells the User-agent not to index any pages on the site)
Disallow: /name-of-microsite/
Disallow: /directory-not-to-index/

Disallow can be used as a negative by entering nothing after the colon. This is essentially an “Allow this crawler to index any and all pages on this website” command. There is only one practical reason that I’ve ever found to use this derivative – where you set a disallow for all user agents, but want to override it for only one agent. To keep it straightforward, come back to this one when you are much more comfortable with this topic later.

Robots.txt: Where to put it

This is a simple answer, but a very important one. Once you finish building your file, upload it right into the root directory of the website. If you place it anywhere else, search engine spiders will consider it to be merely a posted document and not a set of instructions to review prior to crawling.

Difference between robots.txt and the “robots” meta tag

Robots.txt and the robots meta tag are both effective ways to tell search engines not to crawl or index a specific page. I’ve heard many an SEO split hairs about whether there is any good reason to use one over the other.

While this is technically splitting hairs, keep in mind that the robots.txt file is massively more scalable than it’s meta tag cousin. Why? Because you can disallow access to an entire directory on your site with two lines of code. If that directory were to have, say, 18 pages in it, you would have to physically edit, save, and upload the new version of each page to the server individually for the same result.

More Information

I strongly recommend you take the time to learn how to manage this yourself. It is really not very difficult, and something you can keep in your back pocket for later when you really need it.

For those of you who don’t need to do this more than one time, I stumbled upon a robots.txt generator while researching a couple of items for this blog post. If you use it, let me know how it goes. I am sure there are multiple tools out there, and would rather only share the good ones.

In closing, the following is a nice chart from technyat.com that explains all this in an easy-to-understand comparison chart format. Enjoy!

: Image Source: Technyat.com

____________________

Google Plus: What Is It Really All About?

Posted on August 30, 2011 by tpltx70

Most of you have likely found your way onto Google’s latest experiment in social media, Google Plus / Google+. I have been a member since the second week it was available for sign up, but have held off on making commentary here on Return On Now until I could better evaluate it.

Now that we have had several weeks to use and discuss Google+, it’s time to get in on the conversation.

First Impressions of Google+

The thing that immediately jumped out to me, like most of you, was the interesting implementation of circles rather than following in a one- or two-way fashion (a’la Twitter or Facebook). The ability to target postings to a specific circle is a great feature in concept, although it can be a bit cumbersome to use if you are very active on the site. Either way, it’s nice to have the flexibility.

The site comes with a built in +1 button on all posts, comments, and links. That’s great, especially now that Google has built the feature into their algorithm. Now, the +1 works a lot like the Facebook Like button for both liking and sharing. It’s a good thing they caught up on that one quickly.

The Hangout feature looks interesting, but I haven’t used it much due to a lack of time to play around online. It’s probably cool for some purposes, perhaps virtual meet ups or even a round table, so I still need to find time to play with it.

The Photos tab is an interesting view the first time you view it, but not something I find myself ever drawn to click again. And they’ll have to beef up the games section quite a bit. It might be a unique way to play games, but selections are limited at this point in time, mostly titles you can get on an iPhone.

Overall, it’s interesting, but I still struggle to find the motivation to use it, even after several weeks.

Google Plus as a Social Tool

The biggest issue with Google+ is that it comes across more as a “social tool” than as a social network. Google is a analytical company who provides technical products / tools / services for a variety of purposes. Since they felt they had to “be social”, they have been dipping toes in the water for some time now (e.g. the failed Google Buzz experiment). So they built a tool that incorporates a lot of what you see and like on the leading social networks. And they tried to add additional structure to what has been a bit unstructured elsewhere.

I question whether Google understands what a community is. This is a vastly different animal than something like AdWords or Google Docs. Their communication has been sparse, which is a major issue for those who have been singled out by the Real Names Only rule. The kicker is that they simply group Google Plus with all their other services, and if they kick you off for using a fake name, they kick you off everything!

With so many of us drinking the “cloud” Kool-Aid, moving to Google Docs and Microsoft Office 365, this is a massive error on Google’s part. After all, if this is just a “project” as they have been calling it, why would our other services suffer if we are deemed unworthy for their community? This has not affected me since I always use my real name, but others have been burned badly in this fiasco. Sorry Google, but this is a perfect example of the word FAIL.

Google Plus for SEO and website traffic

I would be remiss without considering how Google+ might impact Search Engine Optimization. After all, we are talking about Google and their 90% market share in search.

One thing they are getting right is the +1 button. By making it easier to share on Google Plus from anywhere online, they get the outstanding effect that the Facebook Like button offers. Even more importantly, it gives the public a chance to voice positive votes for content, even with social sites linking as “nofollow“. This is a huge change, and one that should add a lot of value in social search.

In reality, I’m torn between whether Google Plus is truly an attempt at creating a social network or more about figuring out how to meld social data into search. I actually hope it is at least in part about the latter.

Regardless of their motivations, the market for social networks is already saturated. We have invested too much time in the frontrunners, and I am not seeing the social graph of the masses looking to move. Sure, the social media crowd has jumped on full speed ahead, but even among that group, opinions of Google+ range widely.

Final Thoughts

It will be interesting to see if Google figures it out, or if this becomes just another failed social experiment. Or perhaps it offers some unforeseen value for social search. Some of this could already be built into their search algorithm, and all the refinements we’ve seen over the past 6 months very well may represent them still trying to get it right.

What do you think? Is this a real attempt at being social, or a short term experiment to figure out how to meld social activity into SERPs?