Kavliwashere.com https://kavliwashere.com Digital stories, testing and people Fri, 01 Nov 2019 14:03:17 +0200 en-GB hourly 1 https://wordpress.org/?v=5.3 Emoticons are older than I thought https://kavliwashere.com/emoticons-are-older-than-i-thought/ https://kavliwashere.com/emoticons-are-older-than-i-thought/#respond Thu, 21 Feb 2019 06:00:07 +0000 https://kavliwashere.com/?p=3843 I am a closet amateur historian. Even if I, at least according to my friend who is an actual historian, lack the stamina that makes her keep researching exactly when someone built a certain set of railway lines. I’m probably not cut out for the hardcore historian lifestyle (I do spend a grotesque amount of […]

The post Emoticons are older than I thought appeared first on Kavliwashere.com.

]]>

I am a closet amateur historian. Even if I, at least according to my friend who is an actual historian, lack the stamina that makes her keep researching exactly when someone built a certain set of railway lines. I’m probably not cut out for the hardcore historian lifestyle (I do spend a grotesque amount of time going through medieval recipes and dress patterns though).

To nurture my historical interest I follow a blog called Yesterdaysprint on Tumblr. They post small newspaper clippings from the last 200 years or so. A few days ago they posted this wonderful clipping:

Yesterdaysprint Tumblr: Harrisburg Telegraph, Pennsylvania, March 29, 1881

Look at that! That’s some very early emoticons all the way from 1881!  The emoticon got popular in chatrooms in the 1990s, where the :-) and :-( symbols among other symbols helped chatters express feelings and moods. It’s also in this period that “LOL” and other abbreviations got added to the vocabulary of online communicators. 

But this form of communicating go further back. Way back. 

According to the University of Pittsburgh, the oldest smiley we know of is from 1635 and was added next to the signature of the notary going through the accounts of the Town Hall of Trenčín. The little smiley expressed the notary’s satisfaction with the records.

smiley350
The world's oldest known smiley-like image in a financial record archived in Trenčín from 1635 - University of Pittsburgh.

But of course, the emoticon go was back! Of course, there would have been a human being – most likely in a hurry – experimenting with different forms of expressions on the typewriter, and with the morse language, and with the written language. 

In a world where this form of communication mostly has taken place through text messages, it made sense to shorten down messages to symbols and abbreviations to save time. People engaging in the communication have probably given great concern to how typed messages appear neutral and are difficult to translate emotionally. So they added 

Because that’s what humans do. We play with language, communication and the tools we use to communicate with.

I’m also sure that somewhere, somehow, a spirited youth’s typewriter sooner or later would produce the classical representation of a female’s bosom, immortalized on the digital calculator and the early SMS: (.)(.)

There’s also this lovely piece of Ascii art from the roaring 1920s. Just imagine that a person made that on a typewriter after much trial and error.

tumblr_inline_pfg00gnugv1sgr92o_640
Yesterdaysprint Tumblr: The Washington Times, Washington DC, October 2, 1922

It beats writing “58008” on a calculator and turning it upside-down any day.

For a more in-depth story of the emoticon and visual language, I recommend this lovely article: “The History of Emoticons” by Tim Slavin, Off Beat, May 2014

The post Emoticons are older than I thought appeared first on Kavliwashere.com.

]]>
https://kavliwashere.com/emoticons-are-older-than-i-thought/feed/ 0
How I thought flexible work hours would be vs. how it is https://kavliwashere.com/flexible-work-hours/ https://kavliwashere.com/flexible-work-hours/#respond Tue, 05 Jun 2018 13:07:02 +0000 https://kavliwashere.com/?p=3414 About 1½ month ago, I started working flexible work hours, often from home. It means that I’m in charge of my own little work area, and that I get to decide a lot of things. For example when and how I work. It’s been a really interesting experience. I’ve wanted to try something different from […]

The post How I thought flexible work hours would be vs. how it is appeared first on Kavliwashere.com.

]]>

About 1½ month ago, I started working flexible work hours, often from home. It means that I’m in charge of my own little work area, and that I get to decide a lot of things. For example when and how I work.

It’s been a really interesting experience. I’ve wanted to try something different from working 9-5 in a giant office for a very long time. However, I’ve also found out that several of my dreams and hopes of working this way turned out to be more complicated than I thought.

Here are 5 different ideas I had about working like this, and how it has turned out in reality (Hover or tap the cards to flip them).

1. I will get so much more time to exercise because I can do it in the middle of the day​

I will have lots of time to do all the things I didn't have time for before. I'll go swimming, do a daily yoga routine, and go for walks to easen up my desktop-job body.

No, except for the swimming part

Breaking my work flow feels absolutely horrible. If I'm in the middle of a task, I work on that task and forget the time around me. If I want to exercise I either have to do it early in the morning, just before dinner, or some time after dinner late in the evening. Just like before. The only exception is that I can now go swimming when everyone else is working, and that's awesome.

2. I will work less but be more efficient​

I control how I use my day, so I don't have to go to a million stupid meetings all the time. Instead, I will work way more efficient, but use less time working. That will mean more spare time for me!

Not at all

Since I am responsible for my workday and output, I also end up working the entire workday to get as much done as possible. I spend less time on social media and haven't touched my online game that I've played religiously every day for the last 6 years.

3. My diet will be so much better​

Since I won't depend on what is being served in the canteen that day, I can enjoy healthy and nutritious lunches and snacks. I also won't be tempted by all that office cake and candy.

Kind of true

The problem is that I forget the time, so I don't notice that it's lunchtime until my stomach churns and my blood sugar levels are acting up. So I open the fridge, stare into it and, low on sugar, take the first the best lunch I can find. That means I have to stock up on extremely easy lunch options if I'll avoid desperately scavenging the kitchen for some quick energy.

4. I'll work in interesting places to stimulate my mind

I can go wherever I want to, so I'll work from a new and interesting location every day. I'll chill in my garden, go to the park, to museums, and to work collectives.

Nope

Turns out that it's really friggin hard to work on a computer outside. Plus, my home office is just so much better suited for my work. I need a large screen for editing websites and video recordings, a mouse, a keyboard and a comfortable chair. I can't find that at a random café.

5. I'm going to love working skewed hours​

I can finally work when I feel like it. The days of staring at a screen, pretending to work because my tasks are already done or I'm waiting for someone else, are over! No more bad conscience for lacking energy to find new tasks to do.

True!

I really love working when it fits me and my time. And if there's nothing to do because I'm waiting for someone else to finish their task, I just work later. The freedom to go for a walk or a swim in the middle of the day is priceless. And surprisingly, because I'm responsible for my areas, there's always something to do.

So even if most of my ideas didn’t turn out the way I thought it would, it has nonetheless been a lovely experience.

The most surprising thing I’ve learned is that I really need to be strictly organized in order to fulfill my goals of both working, being physically active, and eating well. I mostly don’t succeed. Time just flies by. I’ve considered that perhaps I need actual timers for the different blocks of my day. But timing and planning every minute of my day also defeats the purpose of working flexible.. 

I’ll probably fall into some kind of routine. Or maybe the answer lies in all of those work-life-productivity-upstart-company articles that I find on Medium. One is allowed to hope.

The post How I thought flexible work hours would be vs. how it is appeared first on Kavliwashere.com.

]]>
https://kavliwashere.com/flexible-work-hours/feed/ 0
What is a test persona? https://kavliwashere.com/what-is-test-persona/ Wed, 04 Apr 2018 08:04:39 +0000 http://www.kavliwashere.com/?p=1991 Lightning round: What is a test persona? What is it? A test persona is a fictional person that represents a system’s potential users. Also called “user personas” or simply “personas”. Why do I need it? By identifying and considering a personas  disabilities and needs, you can discover scenarios and problem areas that you never imagined existed. […]

The post What is a test persona? appeared first on Kavliwashere.com.

]]>

Lightning round: What is a test persona?

What is it?

transport (22)

A test persona is a fictional person that represents a system’s potential users. Also called “user personas” or simply “personas”.

Why do I need it?

First impressions

By identifying and considering a personas  disabilities and needs, you can discover scenarios and problem areas that you never imagined existed.

Where is it used?

Personas are valuable when working with any system that is to be used by users. 

Who use it?

Heuristic walkthrough

Personas are often used by user and customer experience (UX/CX) professionals, but others could gain a lot by considering their system users during development and testing.

When is it relevant?

When you need to come up with a system’s different functional flows or possible problem areas, emphatizing with test personas can be a great help.

How do I make one?

drawwireframes

Think about who the users are, why they use the system, and what problems they can encounter. Condense that information to a nice visual format that can be referred to when you work.

Things that can describe a test persona

There are so many things that can be used to describe a user. Find a good balance between a simple and easy to understand test persona, and a test persona with the necessary depth 

  • Personal data: Anything that lets you test that specific user, like customer ID or social security number.
  • Scenarios:What would the user typically do in the system? What areas have to work well?
  • Disabilities: Are there any challenges for users who are colour blind or partially sighted? Can your app be used by a person with very large fingers? It doesn’t have to be large and life-changing disabilities (Though it would be nice to consider those as well)
  • Technological experience: Does your users need a specific knowledge within computers or smartphones to use your system? Do your real life users always have that? If you’re thinking “The user would never do that” then you can bet your life savings that someone will.
  • Name and personality: If everyone in your project “knows” the persona, it’s easier to discuss problem areas around that persona. In one of my project we often talked about “Inga”, a nice elderly lady. Saying “What would Inga do?” helped us see the system in a different light.

You can describe your test personas in a hundred of other ways, like considering needs, demographics, cultural background, environment that the system is used in, and so on. The important thing is to find out what you need, and not add unnecessary details.

Test persona examples

There are hundreds of free templates. Most of them have way too much information to my liking, and are more confusing than helpful. Here are a few examples that I like.

A very simple template from Skcript.com that I really like.

 Test persona made by the Interaction Design Foundation who also have extensive information on personas.

Simple template from  Legaltechdesign.com that incorporates use cases.

The post What is a test persona? appeared first on Kavliwashere.com.

]]>
What is a test plan? https://kavliwashere.com/what-is-a-test-plan/ https://kavliwashere.com/what-is-a-test-plan/#respond Thu, 22 Mar 2018 08:50:39 +0000 http://www.kavliwashere.com/?p=368 A test plan is a tool you use while planning a period of testing. It basically describes what will be tested, who will do it, where they will do it, and how they plan on doing just that.
In some coorperations, where law and contracts are really important, test plans are usually the size of a small child, and nobody ever reads it. Apart from lawyers. That's because test plans aren't always made for you - they can also be made for your team or for different stakeholders.

The post What is a test plan? appeared first on Kavliwashere.com.

]]>

Lightning round: What is a test plan?

What is it?

transport (22)

A test plan describes a planned period of testing for a specific project. It can look like 120 pages of information, or be a quick one-page drawing in a notebook.

Why do I need it?

First impressions

It’s a handy tool for keeping focus and overview, and you can change it as the project proceeds, if you need to.

Where is it used?

In any project where  an overview and plan of the test process is needed. Many companies have their own test plan template that their projects have to use.

Who use it?

Heuristic walkthrough

Often it’s testers and test managers who use the plan, but test plans are made for many people. The project team, project managers, stakeholders, legal department… you name it.

When is it relevant?

A test plan is useful throughout the entire test process. You can plan and estimate test activities from it, or use it later in the process to check up on whether you’ve forgotten something.

How do I make one?

drawwireframes

Make a plan specifically for your project, and consider time used for writing the plan vs time used actually using the plan. It’s supposed to be a help to your project, not a mini-project in itself.

Things you might find in a test plan

  • An overview of people involved in the test process and their responsibilities
  • Calendar overview and important dates
  • An overview of the project’s timeframe and budget
  • Descriptions of different test methods
  • Descriptions on how testing will be carried out and reported
  • Contact information on people that the project need to get in contact with
  • Visual drawings and mind maps that support the test process
  • Technical specifications on the system and software used
  • Notes on anything related to testing the project

A note on standards, best-practices, and certified test plans

You can find many recipes and free test plan templates online. Most of these will tell you that you must have specific data in your test plan because it’s “Industry standards” or “best practices”. 

That’s lazy-talk. The standardized plans cover everything anyone ever wrote in a test plan. You’ll end up with unnecessary information, and that defeats its purpose. 

If there are any legal requirements for your project’s test plan, or if you can use parts of the standardized plans, then by all means follow and use them. But make the document with the people who will actually be using it in mind, not because of formalities.

Test plan examples

The cover-it-all template

The free online template, or the company’s fits-all-projects plan. Remove large parts of it.

The short mind map

A small mind map from one of my own projects. It’s easy to understand, and provides a quick and visual overview of the test period.

The one-pager

I like this 1-page usability test plan template. The structure is a good starting point for any sort of testing really.

The post What is a test plan? appeared first on Kavliwashere.com.

]]>
https://kavliwashere.com/what-is-a-test-plan/feed/ 0
GDPR, Miming passwords and laws in Denmark https://kavliwashere.com/gdpr-miming-passwords/ https://kavliwashere.com/gdpr-miming-passwords/#respond Mon, 12 Mar 2018 08:18:59 +0000 https://kavliwashere.com/?p=2910 There’s a new-ish law in Denmark and the rest of EU. GDPR (General Data Protection Regulation). It contains requirements for how companies are to protect and process personal data. Among other things, companies has to formally describe how they take care of personal data, before they start working with that data. Subsequently, it must also be […]

The post GDPR, Miming passwords and laws in Denmark appeared first on Kavliwashere.com.

]]>
GDPR

There’s a new-ish law in Denmark and the rest of EU. GDPR (General Data Protection Regulation). It contains requirements for how companies are to protect and process personal data.

GDPR

Among other things, companies has to formally describe how they take care of personal data, before they start working with that data. Subsequently, it must also be ensured that they handle their own processes in practice. 

GDPR

Some companies take it very serious. Maybe a little too serious. That led to some rather interesting rules in a project where I worked with personal data – as testers often do.

GDPR

Our test customers were copies of production data. That means that I worked with real peoples’ very real social security numbers. 

GDPR

One day, one of my team members wanted me to go through a flow with a specific customer. The problem was that we didn’t know how I could get the customer’s social security number from her.

GDPR
1

She wasn’t allowed to send me the number over email.

GDPR
2

She couldn’t print it or write it down, since papers with personal information weren’t allowed on our desks. I would also not be able to store the piece of paper somewhere, because I as a consultant wasn’t allowed to have a cabinet or other kinds of furniture.

GDPR
3

She couldn’t say it out loud since we were sitting in an open office.

GDPR

We ended up with a very elegant solution.

GDPR

She sent me the numbers in an encrypted excel file. She then proceeded to mime the encryption key for me (And the rest of the project group).

GDPR

From this experience I learned that it takes a long time to decode mimed passwords that are made up of random letters and numbers.

I also learned that holding up one finger can mean a lot of other things than simply the number “1”.

The post GDPR, Miming passwords and laws in Denmark appeared first on Kavliwashere.com.

]]>
https://kavliwashere.com/gdpr-miming-passwords/feed/ 0
Do’s and don’ts of starting up test automation https://kavliwashere.com/starting-up-test-automation/ https://kavliwashere.com/starting-up-test-automation/#respond Mon, 26 Feb 2018 10:00:21 +0000 http://www.kavliwashere.com/?p=2412 Ah, the promised land of test automation. So much money, time and energy saved! When starting up test automation, your goal is efficiency and frequent checks! Yet, a lot of automation efforts are far from efficient. And that’s too bad, because there really is a lot of good things in test automation – if done […]

The post Do’s and don’ts of starting up test automation appeared first on Kavliwashere.com.

]]>

Ah, the promised land of test automation. So much money, time and energy saved! When starting up test automation, your goal is efficiency and frequent checks! Yet, a lot of automation efforts are far from efficient. And that’s too bad, because there really is a lot of good things in test automation – if done right.

In a pension project, we implemented a solution where health declaration were automatically accepted, denied or put in a queue for manual processing, depending on four different scores. While the solution was implemented, I made an automated data-driven test suite, testing all the different combinations and ranges of the scores. 

The final test, after a month of development, took me 10 minutes. It actually only took me 5 minutes, but the project leader couldn’t believe her ears, so she made me run it again.

That project was perfect for test automation. But not every project is as straightforward. I’ve collected some Do’s and Don’ts for you if you’re considering implementing automated test suites.

Don't...

Automate all your manual tests

Manual tests are amazing, and will find problems you never imagined existed. But the manual approach does not always translate directly to automated checks. Consider making new scenarios specifically for automation.

Leave the automation job to one specialist

While it’s a good idea to hire an automation consultant to get everything started, make sure that the knowledge is shared with others in the project. One day, the consultant or key person will be gone, and others will have to take over the test suite. 

Automate everything

There must be a reason that a test scenario is automated. It has to give some sort of value to the project. Start with the mundane tasks that you are doing every day anyway. That’s where you’ll find the first, immediate payoff of your automation efforts.

Automate for automation's sake

Think about what you need, and what you need right now. If you aim for 80%-100% automation in all projects just because “that’s the big thing and the CEO wants it” or “it sounds good when presenting to the board”, you’re wasting time and money.

Buy the first, the best automation tool

Find a tool that fits your needs, and make sure that the tool can actually do what it promises. Don’t be persuaded by a seller to buy an expensive program that is really smart right now, in this specific project or on this specific platform. And don’t force other projects in your company to use it if they don’t need it.

Think in "passed" and "fail"

You will get cases that “fail”. That doesn’t mean that there’s an error in there. Don’t translate a “Failed” flag to a bug, you need to figure out WHY the test failed. Also consider whether the sharp “Passed/Failed” categories are the right way to report the automated runs. Maybe another classification would be better suited?

Run everything, every time

Think about what the purpose of the test run is. If you don’t need to test a specific area – don’t. Only test functionality that needs testing. Testing everything else is a waste and clutters your reports with irrelevant data. Sure, you can have constantly running checks, but make sure their purpose is clear.

Forget to test your tests

You should get used to the idea of testing your test. An automated test scenario shouldn’t be released unless you’ve seen it fail – several times. Run the test under circumstances where it surely will fail, so you can count on that it will when put into action.

Underestimate the effort it takes to keep an automated suite running

Having an automated test suite is not something that can be started up and then run on its own. There are constant changes in your systems, and the suite needs to be updated and expanded. Don’t leave the responsibility to that one technical tester in your company.

Do...

Break down tests into independent scenarios

It’s easier to figure out where the problems and errors are if your suite consists of short, specific test scenarios. Don’t test it all in one scenario. If you have to look at every single thing that is being checked in the scenario, troubleshooting will be VERY hard. Keep it short and simple.

Make the automated test suite a common project effort

Why not let everyone, not just testers, have an influence on and contribute to the suite? Developers, project manager, product owners.. they all have different reasons for wanting an automated test suite, and different ways of contributing.

Let people know of the results of the automated runs - and think about who the receiver is

Provide different test run results to different roles in the project. A developer will want information from an automated test suite, that a project manager have no interest in. Involve all roles in the project and hear them out on what kind of information they need.

Start out small and create the test suites along the way

In the beginning, create what you need and follow the work flow of the project. Projects change constantly. Don’t waste time, energy and money on creating something that will never be used or is changed in the last minute. Create the suite along the way, bit by bit.

Make your automated setup work hard

You’ve bought the expensive program, you’ve made the setup, you’ve used a lot of time to create and update the test suites.. Use it! Make a suite that can run often AND is valuable to the project. An automated suite that is not running, is a waste of money.

Consider that your project and company changes

Find a program that can grow with your projects and company, and know what you can do – and can’t do – with it. If it isn’t working for you, discard it. There’s no reason to waste your time and money on inadequate tools.

Unburden your manual testers

Try to think of the automated test suite as a helping hand for the testers. Free up their time from tedious, repetitive tasks, so the tester can do what they are best at – testing, not following scripts.

Make your suite quick and easy to run

Setting up and starting automated test runs should be an easy task with some sort of immediate feedback. If your suite is slow or complicated, people won’t bother running them.

Keep the suite constantly updated

The rule is, if your code base changes, then so does your test suite. This is true for ALL code base changes. When bugs are fixed, new features are also added to the system, and that means updating the test suite.

And last, but not least...

Don't just accept what some random article on the Internet tells you

YOU know what you need, so YOU decide whether these Do’s and Don’ts apply to your test setup.

Happy automating!

The post Do’s and don’ts of starting up test automation appeared first on Kavliwashere.com.

]]>
https://kavliwashere.com/starting-up-test-automation/feed/ 0
Testing deep learning chatbots https://kavliwashere.com/testing-deep-learning-chatbots/ https://kavliwashere.com/testing-deep-learning-chatbots/#respond Wed, 14 Feb 2018 09:38:58 +0000 http://www.kavliwashere.com/?p=2423 Chatbots already exist, and have done so for decades. But the classical bots from "ye olden days" have their limits. They rely on a database of thousands of pre-made answers to pre-thought up conversation scenarios. Every time a user writes something, the bot looks through its database for an answer. If a user's question alternate a little from the bot's script, it's followed up by a standard "I don’t understand the question" reply. It's a stone sure way to find out if that sweet girl you're talking to on Tinder is actually a human..

The post Testing deep learning chatbots appeared first on Kavliwashere.com.

]]>

I’m lucky to have gotten involved in a project testing chatbots. It’s an emerging technology, so I’ll do a series of posts on different aspects of testing chatbots.

The problem with simple chatbots...

Chatbots already exist, and have done so for decades. But the classical bots from “ye olden days” have their limits.

They rely on a database of thousands of pre-made answers to pre-thought up conversation scenarios. Every time a user writes something, the bot looks through its database for an answer. If a user’s question alternate a little from the bot’s script, it’s followed up by a standard “I don’t understand the question” reply. It’s a stone sure way to find out if that sweet girl you’re talking to on Tinder is actually a human..

The person behind the typical dating-site bot has programmed it to follow a best-scenario conversation. If the user deviates from the script, the bot is quickly exposed.

It’s nice that it’s so easy to deviate from a bots script if you’re trying to decide whether you’re chatting with a human or not. If you are a customer trying to get support for a problem however, it can be pretty darn frustrating.

Enter deep learning chatbots

“Deep Learning” is a method of making computers able to learn from patterns in conversation data. It’s a part of machine learning, and it’s actually not far from how humans perceive the world.

Imagine that you had to start all over every time you perceived something. You had to consider every little detail about that dirty coffee cup Peter left in the sink. What is it? What’s its’ shape? Is it a threat? What’s it for? There’s something in it, what is that? What are those letters on it? How do I use it? It looks like I can grab it, is it a weapon?

…No one would get anything done.

Luckily, the human brain is exceptional at making assumptions and at remembering how things worked the first time it met it. Instead we’d go “Oh, that scoundrel Peter left his dirty mug in the sink again, just like yesterday”.

testing deep learning

The same thing applies to a deep learning chatbot. It doesn’t have to start from square one every time a user writes a message. It learns from its’ interactions by using “deep neural networks”, a set of algorithms inspired by the human brain. The deep neural networks are designed to interpret sensory data (Text, images, sounds etc.) through a kind of machine perception. It basically makes the computer able to recognizing patterns by labeling or clustering conversation input.

Deep learning is not a new idea. In the 80s and 90s a lot of work was done in the area of machine learning. The difference from then to now is that we have computers with way better processing power, and easier access to much larger amounts of data for the bot to learn from.

Getting a deep learning chatbot is like getting a baby

Just like humans, a deep learning bot has to start somewhere. Without data, the bot is just a bundle of algorithms. The state of a recently released beta chatbot with limited capabilities is also called a cold started chatbot

Generally speaking, the chatbot can learn in two different ways; supervised or unsupervised learning.

Supervised learning

When a bot is taught about how it should classify data by a human.

It’s close to when I tell my daughter about the world. I show her a tomato, and tell her that it’s a “tomato” and that she can eat it.

Chatbots under supervised learning is trained to classify input in a certain way by the people who made the bot (or sometimes even users).

If you’ve chatted with a bot that asked you if some information was the thing you were looking for, you’ve encountered supervised learning in the making. The bots actually asks whether its’ classifications are right or wrong. If the user implies “yes”, the chatbot will remember and suggest the same content the next time a user asks it a similar question.

Unsupervised learning

When the computer is left to its own, free to make its own connections and patterns of the input it receives.

If my daughter were to learn unsupervised, I’d tell her where the fridge is, and let her figure out how the things in it should be categorized and used. Maybe she’d put tomatoes in the same category as oranges, because both things are round. She could also group tomatoes with chili, since both are red.

It would be quicker for my daughter to categorize the content of our fridge this way. She wouldn’t have to consult me. However, her categorization of things in the fridge could just as well be useless to me. If I needed tomatoes for at pasta dish, I wouldn’t agree that tomatoes and oranges are mostly the same because they are both round.

A good example of unsupervised deep learning is Neuralconvo, a chatbot that learned all it’s conversation patterns by analyzing movie scripts. The result? A bot that knows ALL the movie one-liners ever, but outputs absolute gibberish when you try to have a decent conversation with it.

Neuralconvo is a Deep Learning chatbot that makes very little sense.

Unsupervised learning is definitely interesting, and extremely efficient. Most companies would however want some sort of control over how their bot categorizes data and finds information for the customers.

But what about testing deep learning chatbots?!

Some testers are used to testing up against some sort of end-result. They’ve read the requirements and have sketched out all possible test scenarios before testing.

Forget about doing that.

When testing deep learning bots, you don’t really have any pre-decided output for different scenarios to lean on. The bot can answer a customer’s question in various ways, and since the bot is always learning new things, the answers will vary from day to day. 

It’s still possible to test it though. You can always make assumptions about how you expect the system to act. You can also follow the development of the bot from (cold)start to finish. Below are three different, but very important, stages in chatbot development, where testing is important.

1

 The proofreading stage

Before releasing your bot anywhere, test that it’s doing what it was created for.

Run a number of tests where you give it expected input, and see that it returns a fitting output. You should also test any integrations to other systems and platforms.

Since the data that forms the basis of the bot’s learning in this stage is still a rather small sample and under your control, correcting unwanted behavior is possible and easy.

2

The beta stage

Release the bot to a small sample of real-life users and let them beta-test. Every beta tester is valuable as each individual will treat the bot in a unique way. Many users will also find it interesting to explore the boundaries of the bot, and provide it with completely irrelevant inputs.

In the beta test stage the bot will be subjected to unexpected inputs that you never thought of, such as slang, jargon, and colloquial language. And auto-correct. You will quickly find out just how frustrating auto-correct is for your data.

3

The surveillance stage

When the bot has been released into the wild, give users the opportunity to report in problems they encounter. This will give you even more input.

Users should be able to report problems right away, preferably inside the chat. Make your chatbot work for you. Let it gather problems users encountered, and send them directly to you. 

Remember to thank your users every time they submit a problem. Every reported problem and idea make your bot a little better.

A few last words for deep learning testers

When testing deep learning bots, you need to let go of the urge to know every scenario of the system. Testing chatbots is about exploring and experimenting to discover and learn about unexpected data patterns and classifications.

Instead of trying to give your customer a check list of what works and what doesn’t, give them your professional account of how the system you’re testing behaves. Maybe 

The post Testing deep learning chatbots appeared first on Kavliwashere.com.

]]>
https://kavliwashere.com/testing-deep-learning-chatbots/feed/ 0
Chatbot timeline: Stories of chatbots, virtual assistants and conversational interfaces https://kavliwashere.com/chatbot-timeline-history/ https://kavliwashere.com/chatbot-timeline-history/#comments Fri, 02 Feb 2018 09:21:36 +0000 http://www.kavliwashere.com/?p=2397 “Can machines think?” When Mathematician Alan Turing in 1950 asked that question, he also came up with the Turing test, a simple procedure to determine when a computer can “think”.  Let a human chat via Teletype with a computer and another human; if the person can’t determine which is the computer, then it meets Turing’s […]

The post Chatbot timeline: Stories of chatbots, virtual assistants and conversational interfaces appeared first on Kavliwashere.com.

]]>

“Can machines think?”

When Mathematician Alan Turing in 1950 asked that question, he also came up with the Turing test, a simple procedure to determine when a computer can “think”. 

Let a human chat via Teletype with a computer and another human; if the person can’t determine which is the computer, then it meets Turing’s standards for “thinking.”

Today there’s rich opportunity to be fooled by a bot. There’s still a long way to go before we have “thinking” computers, but we’ve come a long way! I love timelines, so here’s a chatbot timeline of some (but not all) bots and virtual assistants through time, who have played a role in the development of the chatbot and A.I.

The Chatbot Timeline

The 1960s

1961- IBM Shoebox

Virtual assistant, voice-based (IBM)

A calculator that could recognize 16 spoken words and the digits 0 through 9 when they were spoken through its microphone. It was basically an extremely basic version of the voice assistants we know today.

Chatbot timeline

1966 - ELIZA

Chatbot (MIT)

A bot programmed to follow different “scripts”, such as a Rogerian psychotherapist. ELIZAs interactions were very limited, and it couldn’t contextualize conversations or recognize keywords.

ELIZA was made to show how superficial communication between man and machine was (And to make fun of psychotherapists). Despite this, ELIZA was the first chatbot to pass as a human.

The 1970s

1972 - PARRY

Chatbot (Kenneth Colby, Stanford University)

PARRY was made to simulate a person suffering from paranoid schizophrenia. It had a conversation strategy, and was therefore much more advanced and ambitious than ELIZA.

Psychiatrists were shown transcripts of conversations with PARRY and actual patients. They guessed that PARRY was a script in 48% of the cases.

The 1980s

Chatbot timeline

1984 - Mark V. Shaney

Synthetic Usenet user (Bruce Ellis, Robe Pike, Don Mitchell)

The bot generated messages in the net.singles Usenet group. It generated messages from text from other posts in the group, using the “Markov chain” technique (Generating smaller text messages by analyzing larger bodies of text)

Many members of single.net thought the bot to be a real person, as it didn’t stand out from other weirdos in the group.

1987 - The knowledge navigator

Early concept of virtual assistant in a smartphone (Apple Computer)

Former Apple CEO John Sculley made several concept demo videos of “The Knowledge Navigator”, a prototype for a smartphone. Different users were shown interacting with the system through a digital butler (With a bow tie).

The concept was criticized for being both unrealistic and for using a human appearance for the bot.

Chatbot timeline
Chatbot timeline

1988 - Thoughts / Jabberwacky

Chatbot (Rollo Carpenter)

It is designed to mimic human interaction and to carry out conversations with users. It is not designed to carry out any other functions.

Its creator believes that it can be incorporated into objects around the home such as robots or talking pets, intending both to be useful and entertaining, keeping people company.

The 1990s

1992 - Dr. Sbaitso

Chatbot (Creative Labs)

A program that came with sound cards for MS-DOS. It showcased the digitized voices the cards were able to produce. Sbaitso was programmed to act like a very basic psychologist.

The quality of the voice was far from lifelike, and Sbaitso’s responses mostly were “WHY DO YOU FEEL THAT WAY?”

Chatbot timeline
Chatbot timeline

1995 - A.L.I.C.E. (Artificial Linguistic Internet Computer Entity)

Chatbot (Richard Wallace)

The program uses an XML Schema called AIML (Artificial Intelligence Markup Language) for specifying the heuristic conversation rules.[4]

Alice has won several Loebner awards and has been the inspiration behind the film “Her”, in which a man falls in love with a chatbot.

1995 - Albert One

Chatbot (Robby Garner)

Albert was designed to mimic the way humans make conversations. It won two awards for being most human at the Loebner awards in the end-90s.

A part of the program was available online in 1995, where it gathered information about what people would actually say to a chatbot.

Chatbot timeline
Chatbot timeline

1996 - Clippit

Virtual assistant (Microsoft)

Clippy shared tips and instructions to the user, based on the users actions in the Office programs.

The thought behind Clippy was that humans react to computers like they do to other humans. So Microsoft included a human-like face in their software. It was wildly unpopular, and has been made fun of by Microsoft later on.

The 2000s

2000 - Ultra HAL

Chatbot / Virtual assistant (Zabaware, Inc)

An information manager that users can interact with via typing or a speech recognition engine. It’s still in development, and has been so since its first mention in 2000.

A forefunner for the virtual assistants that’s a staple of most smartphones today.

Chatbot timeline
Chatbot timeline

2001 - Smarterchild

Chatbot / Virtual assistant (ActiveBuddy Inc)

The first bot made with instant message and social platforms in mind. Smarterchild started out as a word-based adventure game, but soon included instant access to news, weather, stock services, as well as tools like calculators and translators.

The forerunner of Siri, Alexa and other virtual assistants.

2006 - Watson

Question answering computer (IBM)

A program made to understand questions that humans ask and to provide answers that humans can understand and justify.

Watson appeared on Jeopardy, but the prelude wasn’t without challenges. A conflict ensued about whether the show would take advantage of the bots cognitive inferiority. On top of that, Jeopardy ruled that Watson should use the buzzer physically. All was solved with unused questions from earlier shows, and a robotic buzzer-hand for Watson. Watson won first place.

Chatbot timeline

The 2010s

Chatbot timeline

2005 - Mitsuku

Chatbot (Steve Worswick)

Winner of several Loebner awards, Mitsuku was built on ALICE’s framework. It’s still a work in progress, with additions from user generated conversations.

When asked about something, Mitsuku looks up the properties for said thing to find a good reply (E.g. if asked if you can eat a house, it looks up what a house is made of. Since it’s bricks, it will answer that you can’t eat a house)

2010 - Siri

Virtual assistant (Apple / SRI International )

25 years after “The Knowledge Navigator”, Siri was launched, and set the tone for all the other virtual device assistants that followed this decade.

Originally, Siri was not owned by Apple, and was scheduled for release for both Android and blackberry. After only 2 months, Apple bought Siri.

Chatbot timeline
Chatbot timeline

2012 - Google Now

Virtual assistant (Google)

The assistant was developed from the “Google voice search” functionality. It stood out because it could remind users of events without being asked, based on the information Google had on the user. This in turn sparked some discussions about just how much data Google has on its users.

It was originally codenamed “Majel” after the actress who was the voice of computer systems in Star Trek.

2014 - Alexa

Virtual assistant (Amazon)

First made available in the Amazon Echo speaker, Alexa can do a lot of the standard virtual assistant stuff. It can also order things from Amazon, and control smart devices, turning it into a home automation system.

It got its name because the X can be recognized with higher precision. The voice of Alexa was also inspired by the Star Trek franchise.

Chatbot timeline

2014 - Cortana

Virtual assistant, voice-based (Microsoft)

Another virtual assistant, this one is for Windows operating systems. It’s named after the A.I. character “Cortana” from the Halo video games, and voiced by the voice actress from the games.

2016 - Chatbots for Messenger

Chatbots (Facebook / Other companies)

In 2016 Facebook Messenger announced that they would open up for companies to have chatbots on Messenger. 

The first generation of chatbots ranged from being well received but very basic, to extremely bad and unnecessary. The first generation was compared to the first generation of crappy apps made when smartphones caught on big time.

Chatbot timeline
Chatbot timeline

2016 - Tay

Chatbot (Microsoft)

Tay, presented as a curious teenager persona, was made available on Twitter. It made it’s tweets by analyzing text, images and emojis from its interactions with other Twitter users.

It took teh interwebz less than 16 hours to get the bot to post profanities, conspiracy theories, and crude racial remarks. The bot managed to tweet more than 96,000 times before being shut down.

2017 - Bixby

Virtual assistant, voice-based (Samsung)

Samsung also wanted to be a part of the virtual assistant hype, so they made Bixby. It currently resides in my personal phone, and is mean at beat boxing.

Chatbot timeline

The Loebner prize awards

The annual Loebner prize awards has since 1991 held contests for artificial intelligences. The bot considered the most human-like by the judges, win a money prize.

The post Chatbot timeline: Stories of chatbots, virtual assistants and conversational interfaces appeared first on Kavliwashere.com.

]]>
https://kavliwashere.com/chatbot-timeline-history/feed/ 1
Testing chatbots: An introduction to conversational interfaces https://kavliwashere.com/testing-chatbots-introduction/ https://kavliwashere.com/testing-chatbots-introduction/#comments Fri, 19 Jan 2018 08:20:53 +0000 https://kavliwashere.com/?p=2501 Chatbots are already here, but in the next couple of years, we're going to see a lot more of them. Frameworks for them are both affordable and easily adaptable to specific companies. The chatbot can become part of a company's support or information departments, without having to be built from scratch. But how do you test one?

The post Testing chatbots: An introduction to conversational interfaces appeared first on Kavliwashere.com.

]]>

I’m lucky to have gotten involved in a project testing chatbots. It’s an emerging technology, so I’ll do a series of posts on different aspects of testing chatbots.

You probably already know plenty of chatbots

Chatbots, conversational interfaces, digital assistants.. they have many names. A chatbot is, in short, a program that can communicate with a user. The user writes or says something to the chatbot, which then analyzes and finds/calculates an answer to the sentence.

The best known chatbots are probably Amazon’s Alexa, Apple’s Siri, or Google’s.. Google. They are bots that are part of your phone or speaker. They can perform tasks like searching for answers to a question, playing music or ordering something. But you have to ask them in the right way.

The last time I flew with KLM airlines, a chatbot updated me on my flight, and I could ask it to do basic tasks like changing my seat. It worked really well, though it couldn’t help me with more complex tasks, like when my flight got cancelled.

testing chatbots
I only used the KLM chatbot for flight information, but it can also help with travel-related stuff like what to pack for the trip. Clever.

Why use chatbots?

So the chatbots are already here, but in the next couple of years, we’re going to see a lot more of them. Frameworks for them are both affordable and easily adaptable to specific companies. The chatbot can become part of a company’s support or information departments, without having to be built from scratch.

But why would you want to use a chatbot in the first place?

Because chatbots can substitute people! Bam, right in the middle of all the automation-fear that’s going on at the moment.

But even if it sounds a little harsh, the idea is actually quite nice. So let the me rephrase that: Chatbots can unburden people by taking care of the boring, basic routine cases that don’t require an emphatic and brilliant human mind.

Most companies already offer live chat with a human being on their website as an instant contact method for quick questions. A chatbot could easily take care of most of the questions I’ve asked on a website’s live chat channel. Instead of using time and energy on basic tasks, people working in a support department can focus on challenging cases where they make a difference. That makes for happier, more effective workers.

testing chatbots

Testing simple chatbots

When it comes to testing chatbots of the more basic kind, you basically need to test how it answers different questions.

  1. Does the bot know the answers to questions it will encounter.
  2. What does the bot answer to questions it might encounter?
  3. What does the bot answer to questions it probably never will encounter?

Users will probably say “Hi” to the chatbot, so there’s good reason to have that as part of its script, along with all the other highly possible scenarios. In the other end, the chatbot will probably not encounter anyone who starts talking about that their dog just died. But if they do, the chatbot should at least let them know that the question is outside its area of expertise.

testing chatbots
Poncho the weather-chatbot isn’t good for talking about feelings

Sounds simple, right?

It’s not. It’s really content-heavy. Consider all the different ways people might ask a question. Then add spelling errors and auto-correct on top of that. If the questions alternate just the slightest from the bot’s script, users will get the standard “I don’t understand the question” reply. That either makes them close the chatbot, yell, or leaves them desperately trying to figure out the right way to ask. So you need to get some data on what your users are going to ask the chatbot. If you don’t, you’ll be doing a so-called cold-start. Working from that data you slowly and thoroughly add new forms of the same question to the bots database. It’s a tedious job.

Luckily the emerging “deep learning” or “machine learning” technology offers different solutions to that. I’ll take a look at that, and discuss how on earth you can test it, in a following post.

Wanna try out testing chatbots?

It’s always fun to play around and see what their limits are. It gives you a good understanding of what chatbots are and how they function.

  • Mitsuku: Voted most human chatbot in 2013 and 2017. Don’t let the outdated website fool you.
  • Rose – Brilling Understanding Inc: My first attempts at communicating ended up in me being trashtalked. Apart from that, Rose knows a lot about security.
  • Poncho is a weather bot in Messenger. It can tell you about.. well.. the weather.

The post Testing chatbots: An introduction to conversational interfaces appeared first on Kavliwashere.com.

]]>
https://kavliwashere.com/testing-chatbots-introduction/feed/ 1
Creating a demo video in an IT project https://kavliwashere.com/creating-demo-video-project/ https://kavliwashere.com/creating-demo-video-project/#respond Wed, 10 Jan 2018 06:06:19 +0000 http://www.kavliwashere.com/?p=2350 A pension company digitised the flow of receiving, treating and accepting requests for pension products. To the pension workers, what before took place on paper, pen and in ring binders, now became hidden. Though creating a demo video for this project only took a few days compared to the amount of testing I did, it’s importance […]

The post Creating a demo video in an IT project appeared first on Kavliwashere.com.

]]>

A pension company digitised the flow of receiving, treating and accepting requests for pension products. To the pension workers, what before took place on paper, pen and in ring binders, now became hidden.

Though creating a demo video for this project only took a few days compared to the amount of testing I did, it’s importance made a big impression on me.

What was the problem

The new digital flow changed a lot for many people, and few knew what exactly was going on. Communicating the changes to everyone affected was difficult, especially since we there was no time to involve any media department or get access to professional equipment.

  • 3 days (With room for making other things)
  • A broad audience of people ranging from advisors with almost no experience with IT, to IT experts

The mission

To show the current state of the project, and explain the overall digital flow to people working somewhere in it. The new changes had to be clear to a broad audience, from the steering committee, to the IT personnel, to the pension advisors.

story demo video

Gather information about the system, and condense it to a short, easily understandable story.

animation demo video

Create graphics and animations for at demo video that show the flow of the system.

How to do it?

Making the demo video was actually very simple. I used a Powerpoint presentation, my smartphone, screen recording software and a video editor.

create video demo

1. Make a script where you write down all that needs to be said

I made a preliminary script that explained the new flow, in cooperation with the project’s business analysts. They helped with the wording and getting all the details right.

2. make a sound recording of the script

In a quiet meeting room I used my phone to record myself reading the script out loud (It can feel a little weird. Say the script out loud a couple of times to get it right).

3. Create some graphics and animations that supports the script

I created a short slideshow presentation in Power Point with free images I found online. It showed the new flow with images found online. I settled on a reasonable animation speed, and recorded the presentation as it played on my screen.

demo video
The setup of a simple animation made in Power Point

4. Paste it all together in a video editing program.

I made the demo video in Adobe Premiere, but you can use other free alternatives. Import the sound file and the video, and get both to fit each other.

The outcome

I ended up with a very short demo video animating the new flow of a pension product’s way from the customer to the company and back. It was well received.

The demo video was shared on the company’s intranet, so other project groups could see what we worked with. It was also presented to the steering committee, and used to introduce the new system to the pension advisers.

In the project, we were a little surprised with how many people in the project benefited from seeing the demo video. The developers, even though they had made the system, had their focus on the details, and had never seen the overall flow.

The post Creating a demo video in an IT project appeared first on Kavliwashere.com.

]]>
https://kavliwashere.com/creating-demo-video-project/feed/ 0