Published 2022-03-31.
Last modified 2024-08-09.
Time to read: 5 minutes.
posts collection, categorized under Jekyll, Microsoft, SEO, Software-Expert.
I spent a fair bit of time designing the plugins for this Jekyll-powered website for SEO. These plugins are published as open source; click on the word Jekyll in the sentence above this paragraph to learn more. You are welcome!
I do not use Google Analytics because it slows down page load time dramatically. I do use Google Search Console, however, and recently started trying out Microsoft Bing Webmaster Tools.
Microsoft Clarity and Hotjar
The day before April Fool's Day in 2022, I stumbled across Microsoft Clarity, a free product that I knew nothing about. I was curious to know what benefit it might provide.
I learned that one of the things it can do is provide videos of actual user sessions as they interact with the website. Those videos are fantastic.
As a writer of a lot of online content, I have spent hours watching you, dear readers. Individually yet anonymously. Watching people read teaches you a lot about what is important to them. As a writer, knowing your audience allows you to be more relevant. BTW, I do not know who or where my readers are, just the country they are in.
Update 2023-07-10
I no longer need to watch the videos.
Microsoft Clarity's AI generates summaries.
Often it yields incredibly actionable information, but the Key Takeaways section is generally way off the mark.
YMMV.
Today I noticed the insights tab, visible when viewing an entry for a ‘hit’. Seems like Microsoft has employed AI to generate the following impressive summary of the user's behavior. I added punctuation, a light edit and a whimsical pad of notepaper.
Visited URL matches regex: ^https://www\.mslinn\.com/softwareexpert/index\.html(\?.*)?$
Last 7 days
Session insights
Some key takeaways from this session are:
The user was interested in the software expert witness and computer expert services offered by Mike Slinn, as they visited the homepage from Google and spent about four minutes there.
The user was also curious about the technology expert article series, especially the ones related to Git and version control systems. They visited the article index page multiple times and clicked on several articles, spending about 15 minutes in total on this topic.
The user was most engaged with the article on libgit2, a library that provides low-level access to Git operations. They spent about one and a half minutes on this article and clicked on a link to git-fame, a tool that shows the contribution statistics of a Git repository.
The user frequently resized their browser window, which may indicate that they were using a mobile device or adjusting their screen for better readability.
The user also switched between tabs or applications often, as indicated by the page hidden and page visible events. This may suggest that they were multitasking or comparing information from different sources.
01:47 / 47:31
Update 2023-07-18
Today I stumbled across the Clarity Live plugin for the Google Chrome web browser. There is no such plugin for Firefox, sadly.
- GDPR & CCPA ready
- No sampling
- Built on open source
Update 2023-10-04
I just stumbled across the Clarity Client API.
You can quickly get started with Clarity without coding but by interacting with the Clarity client API. This API can help you access advanced features as described in this reference. Add the following calls to Clarity APIs to the HTML or JavaScript of your webpage to access these features.
Note
Your Clarity ID serves as your API key. No other client API key is necessary, and there is no cost for using Clarity client APIs.
Check out this video!
This is one of the first user sessions that Microsoft Clarity recorded for this website.
As you can see, Microsoft Clarity lets me watch movies of users clicking and scrolling through my website; spooky yet very informative.
Clarity is open source. Here is the GitHub project.
I have since learned that Hotjar is similar to Microsoft Clarity.
Online Behavior Matches Real-World Behavior
The user in the above video read a bit about my experience as a software expert witness, then straightaway downloaded my resume. They were on the website for just over one minute. Although they did call me, their online behavior showed a lack of urgency, and their ‘real-world’ behavior mirrored what I saw in the movie.
A week later another party visited my site, and spent 80 minutes carefully reading 3 pages, among others. Their ‘real-world’ behavior also matched their online behavior, in that they exhibited a sense of urgency towards engaging a software expert.
Tracking Downloads And Other Behavior
Microsoft Clarity does not consider a download as a click. It does not even notice downloads. For me, downloads are what I care about most. If you never download my resume, you probably are not a candidate for hiring me as a software expert. I want to track downloads, not clicks. Clicks are nice, but there is a direct relationship between resume downloads and signing contracts with legal firms who represent new clients.
AWS CloudTrail and CloudWatch can provide download details and much more. For example, user IP addresses and geographic location can be captured when a monitored event occurs.
Many Websites Perform Surveillance
Wired Magazine published an article on a similar type of surveillance (keyloggers) on May 11, 2022. I paraphrased two sentences from that article:
- 1.8% of websites studied gathered an EU user's email address without their consent, and a staggering 2.95% logged a US user's email.
- For US users, 8.4% of sites may have been leaking data to Meta, Facebook’s parent company, and 7.4% of sites may be impacted for EU users.
Yours truly does nothing of the sort.
Popular Pages
The data that Clarity manages can be downloaded with the
Clarity Data Export API.
API calls are limited to 10 calls per project, per day; that is not a problem.
The endpoint is www.clarity.ms/export-data/api/v1/project-live-insights.
An access token is required to use the API.
To get one, open the Clarity project and select
Settings -> Data Export -> Generate new API token.
I called my token clarity_data_export and stored it in an environment
variable called CLARITY_DATA
Here is how I retrieved the information about the most popular pages in the
last 30 days using the bash command line.
This bash script requires jq to be installed:
#!/bin/bash DAYS=30 DIM1="PopularPages" PARAMS="\numOfDays=$DAYS&dimension1=$DIM1" # Backslash escape URL="https://www.clarity.ms/export-data/api/v1/project-live-insights" # curl -s \ # --trace - \ # --location "$URL?$PARAMS" \ # --header 'Content-Type: application/json' \ # --header "Authorization: Bearer $CLARITY_DATA" | less # exit JSON=$(curl -s \ --location "$URL?$PARAMS" \ --header 'Content-Type: application/json' \ --header "Authorization: Bearer $CLARITY_DATA" ) echo "$JSON" | jq -r '.[] | select(.metricName == "PopularPages").information[].url'
The result is something like the following:
https://www.mslinn.com/blog/2023/09/14/boost.html https://www.mslinn.com/blog/2021/02/11/javascript-named-arguments.html https://www.mslinn.com/blog/2023/08/20/pytest.html https://www.mslinn.com/blog/2022/01/10/wsl-backup.html https://www.mslinn.com/av_studio/640-music21.html https://www.mslinn.com/llm/4000-wsl.html https://www.mslinn.com/blog/2021/04/28/buildah-podman.html https://www.mslinn.com/blog/2020/10/27/installing-a-new-ssh-key-on-awc-ec2-with-user-data.html https://www.mslinn.com/av_studio/570-ableton-push-standalone.html https://www.mslinn.com/git/600-partial-clone.html
Looking Ahead
I would very much like to receive a continuous data feed of the above AI-generated per-visit summary, in real time as it becomes available. An email stream would be a quick way to start. A streaming api for data would be a logical next step.
Perhaps the Clarity API might be a place to start.
