WEBVTT

1
00:00:00.225 --> 00:00:07.950
You're listening to the day I fixed my site's visibility after discovering it was serving an outdated prototype and lacked analytics.

2
00:00:07.950 --> 00:00:11.125
Written by Angel Nee-evess on Promptway.

3
00:00:11.685 --> 00:00:17.060
Last week I discovered that my own publication was invisible on its own domain.

4
00:00:17.360 --> 00:00:18.047
Not invisible in

5
00:00:18.047 --> 00:00:20.310
the poetic, nobody-reads-my-blog way.

6
00:00:20.310 --> 00:00:21.510
Literally invisible.

7
00:00:21.510 --> 00:00:28.760
promptway.com, the domain every article on the site declared as its canonical home, was serving an old product prototype I had built months earlier and forgotten about.

8
00:00:28.760 --> 00:00:31.610
The actual publication lived on a temporary platform U R L.

9
00:00:31.610 --> 00:00:34.510
Every AI crawler that found an article was being told "the real

10
00:00:34.510 --> 00:00:34.860
copy

11
00:00:34.860 --> 00:00:35.072
lives

12
00:00:35.072 --> 00:00:35.285
over

13
00:00:35.285 --> 00:00:35.760
there,"

14
00:00:35.760 --> 00:00:35.885
and

15
00:00:35.885 --> 00:00:36.460
over there was a

16
00:00:36.460 --> 00:00:36.760
different

17
00:00:36.760 --> 00:00:37.885
product.

18
00:00:38.235 --> 00:00:38.372
I

19
00:00:38.372 --> 00:00:38.535
had

20
00:00:38.535 --> 00:00:38.760
written

21
00:00:38.760 --> 00:00:38.860
a

22
00:00:38.860 --> 00:00:39.147
whole

23
00:00:39.147 --> 00:00:39.510
piece

24
00:00:39.510 --> 00:00:39.822
about

25
00:00:39.822 --> 00:00:40.510
optimizing

26
00:00:40.510 --> 00:00:40.672
your

27
00:00:40.672 --> 00:00:40.985
site

28
00:00:40.985 --> 00:00:41.147
for

29
00:00:41.147 --> 00:00:41.597
AI

30
00:00:41.597 --> 00:00:42.110
agents

31
00:00:42.110 --> 00:00:42.247
and

32
00:00:42.247 --> 00:00:42.522
L

33
00:00:42.522 --> 00:00:42.797
L

34
00:00:42.797 --> 00:00:43.085
Ms.

35
00:00:43.085 --> 00:00:43.247
The

36
00:00:43.247 --> 00:00:43.510
eight

37
00:00:43.510 --> 00:00:43.960
layers

38
00:00:43.960 --> 00:00:44.085
in

39
00:00:44.085 --> 00:00:44.310
that

40
00:00:44.310 --> 00:00:44.785
article

41
00:00:44.785 --> 00:00:44.972
were

42
00:00:44.972 --> 00:00:45.297
all

43
00:00:45.297 --> 00:00:46.885
real and all working.

44
00:00:46.885 --> 00:00:51.135
And none of it mattered, because the front door had the wrong address on it.

45
00:00:51.460 --> 00:00:54.222
So I spent a day fixing the unglamorous parts.

46
00:00:54.222 --> 00:00:56.722
This is the checklist, in plain words.

47
00:00:56.722 --> 00:01:01.285
None of it requires you to be a developer, though some steps need one for an hour or two.

48
00:01:01.285 --> 00:01:07.835
If part one was about making your site readable to machines, this is about making sure the machines ever show up.

49
00:01:08.110 --> 00:01:13.960
Step 0: check what your domain actually serves Open a private browser window and type your domain.

50
00:01:13.960 --> 00:01:14.810
Not your bookmark.

51
00:01:14.810 --> 00:01:16.135
The domain.

52
00:01:16.410 --> 00:01:25.047
Then check three U R Ls by hand: yourdomain.com/sitemap.xml, yourdomain.com/feed.xml, and yourdomain.com/robots dot text.

53
00:01:25.047 --> 00:01:28.447
If any of them 404, that is your whole afternoon right there.

54
00:01:28.447 --> 00:01:29.210
Mine did.

55
00:01:29.210 --> 00:01:35.185
The publication had all three, but the domain was attached to the wrong project, so the live internet got none of them.

56
00:01:35.510 --> 00:01:38.910
While you are in there, settle the www question.

57
00:01:38.910 --> 00:01:47.310
Pick one form of your domain, either yourdomain.com or www.yourdomain.com, and make the other one redirect to it permanently.

58
00:01:47.310 --> 00:01:51.222
Search engines treat them as two different sites until you do.

59
00:01:51.534 --> 00:02:06.509
I picked the bare domain, made www redirect to it, and also made the hosting platform's free preview U R L redirect home, because a copy of your site living on a platform subdomain is a duplicate-content problem you are choosing to keep.

60
00:02:06.784 --> 00:02:12.534
Step 1: let the right robots in Your robots dot text file is the bouncer at the door.

61
00:02:12.534 --> 00:02:17.734
The guest list changed a lot recently, and most sites are still working from an old one.

62
00:02:18.009 --> 00:02:25.446
The crawler that decides whether ChatGPT search cites you is called OAI-SearchBot, and it did not exist when most robots dot text files were written.

63
00:02:25.446 --> 00:02:28.721
Same story for Claude-SearchBot, Perplexity-User, and a handful of others.

64
00:02:28.721 --> 00:02:31.734
If your file does not mention them, you are relying on default behavior.

65
00:02:31.734 --> 00:02:38.434
I would rather be explicit: I allow every major AI search crawler by name, because being cited is the whole point of a publication.

66
00:02:38.759 --> 00:02:40.496
This is a ten-minute edit.

67
00:02:40.496 --> 00:02:48.284
Ask whoever manages your site to compare your allowlist against a current guide to AI user agents and add what is missing.

68
00:02:48.559 --> 00:02:53.146
Step 2: put your whole article in the feed Here is one I had completely wrong.

69
00:02:53.146 --> 00:02:56.359
My R S S feed only carried summaries.

70
00:02:56.659 --> 00:03:00.484
A summary-only feed feels safe, like you are protecting the full text.

71
00:03:00.484 --> 00:03:03.596
What it actually does is cripple every downstream channel.

72
00:03:03.596 --> 00:03:06.446
Feed readers show your readers a teaser and a link.

73
00:03:06.446 --> 00:03:11.121
Syndication platforms that import via R S S get nothing worth importing.

74
00:03:11.121 --> 00:03:15.709
AI systems that ingest feeds get a paragraph instead of the article.

75
00:03:16.034 --> 00:03:19.059
Full-content feeds are how machines subscribe to you.

76
00:03:19.059 --> 00:03:26.384
The fix is technical but small: your R S S feed should carry the complete rendered article, and if you have a jay-son Feed, it should too.

77
00:03:26.384 --> 00:03:29.234
Your developer will know this as content:encoded.

78
00:03:29.234 --> 00:03:42.896
The conversation takes one sentence: "make our feeds full-content." Step 3: tell the engines when you publish, do not wait to be found By default, publishing works like this: you post, and then you wait for a crawler to wander by.

79
00:03:42.896 --> 00:03:44.759
That can take days.

80
00:03:45.034 --> 00:03:47.184
Two free standards flip it to a push model.

81
00:03:47.184 --> 00:03:53.834
IndexNow is a single notification you send when a page is new or updated, and one submission covers Bing, Yandex, and a few other engines at once.

82
00:03:53.834 --> 00:03:59.684
Bing matters more than its search share suggests, because most of the web results ChatGPT cites overlap heavily with Bing's top results.

83
00:03:59.684 --> 00:04:06.784
WebSub does the same job for feed readers: it pings a hub when your feed changes, and subscribers update within seconds instead of whenever they next poll.

84
00:04:07.109 --> 00:04:08.509
Both are set-and-forget.

85
00:04:08.509 --> 00:04:14.509
We wired ours into a small script that runs automatically on publish, and I never think about it.

86
00:04:14.509 --> 00:04:21.096
If you publish through a platform like Ghost or WordPress, there is a decent chance a plugin or setting already does this.

87
00:04:21.096 --> 00:04:22.634
Turn it on.

88
00:04:22.909 --> 00:04:30.021
Step 4: put a human name on AI-assisted work Some of the writing on Promptway is drafted by AI personas.

89
00:04:30.021 --> 00:04:33.434
That is disclosed openly, profile pages and all.

90
00:04:33.434 --> 00:04:43.809
But disclosure alone is not enough anymore, and the sites that got hammered by Google's scaled-content crackdowns had one thing in common: nobody human was accountable for the words.

91
00:04:44.109 --> 00:04:50.434
So every persona-written article on the site now carries a visible line: "Reviewed by Angel Nee-evess," linking to my profile.

92
00:04:50.434 --> 00:04:54.359
It is the same pattern medical sites have used for years with "medically reviewed by Dr.

93
00:04:54.359 --> 00:05:04.334
X." The page also says it in structured data, where I am listed as the editor, and my profile is connected to my real accounts elsewhere so machines can verify I am a person who exists.

94
00:05:04.709 --> 00:05:09.334
One honest note: this is an accountability signal, not a ranking cheat code.

95
00:05:09.334 --> 00:05:14.634
It tells readers and crawlers that a named human stands behind the work, which happens to be true.

96
00:05:14.634 --> 00:05:17.909
If it were not true, I would fix that first.

97
00:05:18.184 --> 00:05:24.309
Step 5: never ship a blank share card Paste one of your article links into Slack or iMessage.

98
00:05:24.309 --> 00:05:29.234
If the preview is a gray rectangle, you are losing clicks you already earned.

99
00:05:29.534 --> 00:05:35.009
Most articles on my site have generated hero art, but a few do not, and those shipped with no preview image at all.

100
00:05:35.009 --> 00:05:40.834
The fix was a small template that auto-generates a clean, typographic card with the article title for any post without art.

101
00:05:40.834 --> 00:05:42.971
Every link now unfurls into something.

102
00:05:42.971 --> 00:05:49.159
Most platforms have this built in; if yours is custom, it is an afternoon of work for a developer, once, forever.

103
00:05:49.434 --> 00:06:04.996
Step 6: measure the AI traffic separately Here is the stat that convinced me to finally set up analytics: visitors arriving from AI tools like ChatGPT and Perplexity are a tiny slice of traffic, around one percent industry-wide, but they convert far better than organic search visitors.

104
00:06:04.996 --> 00:06:09.659
Someone who clicks through from an AI answer chose your site after the machine already summarized you.

105
00:06:09.659 --> 00:06:12.209
That is a warm lead, not a drive-by.

106
00:06:12.484 --> 00:06:15.784
If you lump that traffic in with everything else, you will never see it.

107
00:06:15.784 --> 00:06:24.521
The setup is one custom channel group in Google Analytics that buckets referrals from chatgpt.com, perplexity.ai, claude.ai, and gemini.google.com.

108
00:06:24.521 --> 00:06:31.584
Ten minutes in the admin panel, and from then on you can answer "is any of this AI optimization work doing anything?" with a number.

109
00:06:31.859 --> 00:06:37.859
The part where I admit a fleet of agents did the typing Full transparency: I did not hand-write most of this.

110
00:06:37.859 --> 00:06:42.146
I described the plan, and a team of AI coding agents executed it, each sized to its task.

111
00:06:42.146 --> 00:06:44.834
Small models refreshed the robots file and the redirects.

112
00:06:44.834 --> 00:06:47.759
Bigger ones rebuilt the feeds and wrote the publish-time automation.

113
00:06:47.759 --> 00:06:55.909
I reviewed the output, caught a few real mistakes (one agent excluded the exact files we needed redirected, another drew a logo with the text colliding), fixed them, and shipped.

114
00:06:56.234 --> 00:06:58.184
That division of labor felt right.

115
00:06:58.184 --> 00:07:03.034
The judgment calls, what to build, what to skip, what counted as done, stayed with me.

116
00:07:03.034 --> 00:07:05.284
The typing did not have to.

117
00:07:05.559 --> 00:07:12.521
The whole checklist, one more time Type your domain into a private window and make sure it serves the thing you want found, on one canonical host.

118
00:07:12.521 --> 00:07:16.284
Check sitemap.xml, feed.xml, and robots dot text by hand.

119
00:07:16.284 --> 00:07:17.559
Fix the 404s.

120
00:07:17.559 --> 00:07:21.184
Update your robots dot text allowlist for the twenty twenty six crawler landscape.

121
00:07:21.184 --> 00:07:22.434
Make your feeds full-content.

122
00:07:22.434 --> 00:07:26.084
Wire up IndexNow and WebSub so publishing pushes instead of waiting to be pulled.

123
00:07:26.084 --> 00:07:29.496
Put a named, linkable human reviewer on anything AI-assisted.

124
00:07:29.809 --> 00:07:34.171
Generate a fallback share image so no link ever unfurls blank.

125
00:07:34.171 --> 00:07:37.634
Build an AI-referrals channel group in your analytics.

126
00:07:38.034 --> 00:07:39.221
None of this is clever.

127
00:07:39.221 --> 00:07:40.384
That is the point.

128
00:07:40.384 --> 00:07:47.671
The clever work, the writing, the structured data, the L L M S dot text file, only pays off after the plumbing is sound.

129
00:07:47.671 --> 00:07:49.834
Check the front door first.

130
00:07:50.395 --> 00:07:51.245
That's the story.

131
00:07:51.245 --> 00:07:54.157
You'll find every source linked at the end of the article.

132
00:07:54.157 --> 00:07:56.820
Thanks for listening, on Promptway.