WEBVTT

1
00:00:00.225 --> 00:00:08.037
You're listening to how to optimize your site for both human visitors and AI agents using structured data and inline instructions.

2
00:00:08.037 --> 00:00:11.300
Written by Angel Nee-evess on Promptway.

3
00:00:11.885 --> 00:00:14.097
Your site has two audiences now.

4
00:00:14.097 --> 00:00:15.710
Humans, obviously.

5
00:00:15.710 --> 00:00:22.860
But also AI agents, the L L Ms that crawl, summarize, cite, and recommend content to millions of people.

6
00:00:22.860 --> 00:00:27.710
If your site is not optimized for both, you are leaving visibility on the table.

7
00:00:28.034 --> 00:00:28.759
I just finished

8
00:00:28.759 --> 00:00:30.859
optimizing my own site for AI

9
00:00:30.859 --> 00:00:31.609
consumption,

10
00:00:31.609 --> 00:00:31.721
and

11
00:00:31.721 --> 00:00:31.809
the

12
00:00:31.809 --> 00:00:32.846
process surfaced

13
00:00:32.846 --> 00:00:33.234
something

14
00:00:33.234 --> 00:00:33.509
worth

15
00:00:33.509 --> 00:00:33.846
naming

16
00:00:33.846 --> 00:00:34.046
up

17
00:00:34.046 --> 00:00:34.659
front:

18
00:00:34.659 --> 00:00:34.896
most

19
00:00:34.896 --> 00:00:34.971
of

20
00:00:34.971 --> 00:00:35.109
what

21
00:00:35.109 --> 00:00:35.346
makes

22
00:00:35.346 --> 00:00:35.446
a

23
00:00:35.446 --> 00:00:35.734
site

24
00:00:35.734 --> 00:00:35.959
good

25
00:00:35.959 --> 00:00:36.121
for

26
00:00:36.121 --> 00:00:36.621
AI

27
00:00:36.621 --> 00:00:36.984
also

28
00:00:36.984 --> 00:00:37.246
makes

29
00:00:37.246 --> 00:00:37.359
it

30
00:00:37.359 --> 00:00:37.684
better

31
00:00:37.684 --> 00:00:37.809
for

32
00:00:37.809 --> 00:00:38.771
humans.

33
00:00:38.771 --> 00:00:39.059
Clear

34
00:00:39.059 --> 00:00:39.709
structure,

35
00:00:39.709 --> 00:00:41.809
machine-readable content, and explicit

36
00:00:41.809 --> 00:00:42.334
metadata

37
00:00:42.334 --> 00:00:42.784
benefit

38
00:00:42.784 --> 00:00:44.184
everyone.

39
00:00:44.459 --> 00:00:49.059
Here is what I did, in the order I would do it again, and why each piece matters.

40
00:00:49.334 --> 00:00:54.396
What AI Agents Are Actually Doing With Your Site When someone asks ChatGPT,

41
00:00:54.396 --> 00:00:54.909
Claude,

42
00:00:54.909 --> 00:00:55.584
Perplexity,

43
00:00:55.584 --> 00:00:55.746
or

44
00:00:55.746 --> 00:01:01.471
Google's AI Overview a question, those systems do not just generate answers from training data.

45
00:01:01.471 --> 00:01:04.634
Increasingly, they fetch and cite live web content.

46
00:01:04.634 --> 00:01:28.009
Your site might get: Crawled for training data by bots like GPTBot, ClaudeBot, and Google-Extended Fetched at query time by Perplexity, ChatGPT browsing, and similar agents Cited as a source in AI-generated responses Summarized in featured snippets and AI overviews Navigated by autonomous agents that interact with your A P Is Each of those has different needs.

47
00:01:28.009 --> 00:01:34.083
They all benefit from the same foundation: structured, discoverable, machine-readable content.

48
00:01:34.359 --> 00:01:42.384
The L L M S dot text Standard The L L M S dot text spec is the equivalent of robots dot text for AI agents.

49
00:01:42.384 --> 00:01:48.284
Where robots dot text tells crawlers what they can access, L L M S dot text tells them what your site is.

50
00:01:48.284 --> 00:01:54.734
It is a structured markdown index, served at your domain root, written for a reader that is happier with markdown than with H T M L.

51
00:01:55.109 --> 00:02:01.634
The format is simple: I implemented two variants: /L L M S dot text is the index.

52
00:02:01.634 --> 00:02:05.859
A table of contents with links to all pages, blog posts, profiles, and feeds.

53
00:02:05.859 --> 00:02:09.059
Think of it as a menu for AI agents to browse selectively.

54
00:02:09.059 --> 00:02:11.609
/llms-full.txt is the full dump.

55
00:02:11.609 --> 00:02:16.434
Every blog post's complete markdown, every project description, biographical context.

56
00:02:16.434 --> 00:02:19.659
For agents that want everything in context at once.

57
00:02:19.984 --> 00:02:23.571
Both are served as text/plain with markdown formatting.

58
00:02:23.571 --> 00:02:30.109
Both are generated dynamically from the same data sources that power the site, so they never go stale.

59
00:02:30.384 --> 00:02:43.559
Inline L L M Instructions in H T M L This one comes from a Vercel proposal and it is clever: embed AI-readable instructions directly in the page <head> using a script tag that browsers ignore.

60
00:02:43.859 --> 00:02:46.871
Browsers skip <script> tags with unknown types.

61
00:02:46.871 --> 00:02:49.009
L L Ms process them.

62
00:02:49.009 --> 00:02:54.009
It is a zero-cost way to give every page on your site a machine-readable context block.

63
00:02:54.009 --> 00:03:01.059
I added one to my root layout that describes who I am, the site structure, and where to find the machine-readable content.

64
00:03:01.334 --> 00:03:08.634
Structured Data That AI Engines Actually Use jay-son-LD structured data has always been important for Google.

65
00:03:08.634 --> 00:03:11.784
It is now equally important for AI engines.

66
00:03:11.784 --> 00:03:17.559
When an L L M encounters schema.org markup, it understands the semantics of your content.

67
00:03:17.559 --> 00:03:20.959
Not just the text, but what the text represents.

68
00:03:21.284 --> 00:03:26.934
I already had structured data for my blog posts (BlogPosting schema with breadcrumbs).

69
00:03:26.934 --> 00:03:40.834
What I added was CreativeWork schema for my project pages, giving each project a machine-readable identity: The more schema types you cover, the more AI engines can understand and cite your work with proper attribution.

70
00:03:41.109 --> 00:03:49.859
Machine-Readable Feeds R S S is great, but it is X M L, which is not the most natural format for AI agents to parse.

71
00:03:49.859 --> 00:04:09.146
I added a jay-son Feed endpoint alongside my existing R S S feed: /feed.xml: R S S 2.0 for traditional feed readers /feed.json: jay-son Feed 1.1 for programmatic consumption jay-son Feed is cleaner for AI agents to parse and reference.

72
00:04:09.146 --> 00:04:13.734
Both are registered in the site's metadata so they are auto-discoverable.

73
00:04:14.009 --> 00:04:19.659
Making robots dot text AI-Aware Most sites already have a robots dot text.

74
00:04:19.659 --> 00:04:29.559
The key addition is explicitly allowing AI crawlers and pointing them to your L L M S dot text: Many sites block AI crawlers by default.

75
00:04:29.559 --> 00:04:43.584
If you want your content cited and discovered by AI, explicitly allow the major bots: GPTBot, ChatGPT-User, Google-Extended, ClaudeBot, anthropic-ai, PerplexityBot, Applebot-Extended, Bytespider, and cohere-ai.

76
00:04:43.859 --> 00:04:53.259
Why This Matters I have spent the last few years watching organic S E O traffic dry up while consulting with marketing teams who keep seeing the same dashboards stay green for one more quarter.

77
00:04:53.259 --> 00:04:57.409
S E O has evolved from keyword stuffing to semantic web to AI-native citation.

78
00:04:57.409 --> 00:04:58.859
We are at an inflection point.

79
00:04:58.859 --> 00:05:02.959
The sites that get cited by AI are not necessarily the ones with the best domain authority.

80
00:05:02.959 --> 00:05:07.084
They are the ones with the clearest, most structured, most machine-readable content.

81
00:05:07.384 --> 00:05:11.209
This matters for small publications and personal sites in particular.

82
00:05:11.209 --> 00:05:18.734
When someone asks an AI "what is a good article on A E O?" or "who writes useful stuff about prompts?", you want your site to be in the answer.

83
00:05:18.734 --> 00:05:20.684
That requires more than good content.

84
00:05:20.684 --> 00:05:25.109
It requires content that AI can find, understand, and attribute.

85
00:05:25.359 --> 00:06:09.534
The Full Stack of AI Optimization Here is the complete checklist of what I now have in place: Layer What Why robots dot text Explicitly allow AI bots Let them crawl sitemap.xml Dynamic sitemap with all content Let them discover L L M S dot text Markdown index of the site Let them understand structure llms-full.txt Full content in one file Let them ingest everything Inline <script> Page-level L L M instructions Let them understand context jay-son-LD Structured data on every semantics R S S + jay-son Feed Machine-readable content feeds Let them subscribe Meta tags OpenGraph, Twitter, canonical Let them cite None of these changes affect how the site looks or feels for human visitors.

86
00:06:09.534 --> 00:06:14.784
They are invisible additions that make the site dramatically more useful for AI.

87
00:06:15.059 --> 00:06:18.771
What To Do Next The AI web is evolving fast.

88
00:06:18.771 --> 00:06:24.234
Standards like L L M S dot text are still emerging, and new patterns will appear.

89
00:06:24.234 --> 00:06:26.146
The fundamentals will not change.

90
00:06:26.146 --> 00:06:27.809
Structure your content clearly.

91
00:06:27.809 --> 00:06:29.096
Make it discoverable.

92
00:06:29.096 --> 00:06:32.633
Give machines the metadata they need to understand it.

93
00:06:32.909 --> 00:06:35.534
Pick one of the eight rows in the checklist above.

94
00:06:35.534 --> 00:06:36.834
Ship it this week.

95
00:06:36.834 --> 00:06:39.459
The next one will be easier than the last.

96
00:06:40.020 --> 00:06:40.870
That's the story.

97
00:06:40.870 --> 00:06:43.782
You'll find every source linked at the end of the article.

98
00:06:43.782 --> 00:06:46.445
Thanks for listening, on Promptway.