work

Where did that 15 minutes go?

Open browser to look something up on intranet.

See something funny in Helpdesk system.

Think about tweeting to a friend about it.

Open twitter.

Get distracted by shinies.

Realize you haven’t checked Facebook today.

Open Facebook.

Look at a cute picture.

Realize that you’re not getting work done.

Close browser.

Open email.

Remember why you were looking at intranet.

Open browser.

Notice funny ticket in queue.

Remember why you got distracted in the first place.

Compose tweet about it.

Tweet is too long.

Consider deleting it and getting back to work.

Make blog post instead.

Report Testing Guide for End Users

This is a basic testing guide that I prepared after searching fruitlessly for a guide to give the system users at work. Originally posted here.

Please feel free to share your thoughts about how this can be improved, expanded, made more generic, adapted for other uses, etc.


Why do I need to test?

Testing is essential to making sure that any report or system enhancement is accurate and works as expected. The IT team tests as thoroughly as possible, however there are several reasons that testing should be done by the end user as well:

  1. You are much more familiar with your piece of the system than IT is. SAP is a new system for all of us. While you have been learning how your processes function in SAP, we have been providing cut-over support, learning the data structures, and learning how to look up limited pieces of the data.

  2. YOU are the expert in your area of the business. The IT team has limited familiarity with how you use the system. We are constantly learning where various data elements are and how things interact, but we don’t yet know enough about the operations of any business area to consider ourselves experts. YOU are the expert.

  3. Customer satisfaction is important, and YOU are our customer. You are our customer, and it is important to us that the solutions that we provide meet your expectations and needs.

Types of Testing

Testing can vary quite a bit in complexity depending upon the project and the stage of development. There are three essential types of testing for reports:

  1. Interface Review: Validating that the search criteria and columns included in the report are correct and working as expected.

  2. Data Validation: Detailed review of the data presented in the report in comparison to the source system (SAP) to identify any fields that may be incorrect.

  3. Usability Testing: Testing the report as you would normally use it. This testing can help uncover issues in presentation or in the data included in the report that were overlooked in the Interface review and Data Validation.

Identifying Errors

If any problems are found encountered for any of the tests, they need to be documented and sent to the IT team for review.

For problems encountered during Interface Review:

  1. The exact search criteria used
  2. If you received an error: what that error was and instructions for how to reproduce the error. (Screenshots are always welcome!)
  3. If there was a problem with the results, details about how the results differed from expectation.

For problems encountered during Data Validation:

  1. Example transaction
  2. Which column is incorrect
  3. The incorrect data shown
  4. What the correct data should actually be
  5. Where to see the correct data in SAP.

For problems encountered during Usability Testing:

  1. If a data issue is found, then provide the information listed under Data Validation.
  2. If another issue is found, then outline what the issue is and the requested change.

Report Testing Checklist

Interface Review:

#

Test

Pass/Fail

Problems Encountered

1

Are all required columns present?

2

Are all column names correct?

3

Is the sort order for the results correct?

4

Are all required search criteria present?

5

Are all search criteria names correct?

6

Are default search criteria appropriate?

7

Do all search criteria work properly?

8

Are there any other issues or changes needed?

Data Validation:

The following apply to EVERY column.

#

Test

Pass/Fail

Problems Encountered

1

Is the data format correct?

2

Is there data in every column that should contain data?

3

Validate the data shown on the report: If the report contains multiple types of transaction (for example: notification types or billing document types), check at least two of each. Compare the report to what is shown on the transaction(s) in SAP.

4

Validate calculations: If the report contains any calculations (sub-total, days past due, etc), validate that against the other data on the report and against data from SAP.

5

If this report replaces an existing report or data pull that you use, then compare results between the two reports. Do you notice any issues?

6

Are there any other issues or changes needed?

Usability Testing:

Consider how you would use the report in the normal course of business, then test the report as if it were considered live.

Warning: Do not share results from the report or base decisions upon the report unless you have completed Data Validation testing without any issues.

#

Test

Pass/Fail

Problems Encountered

1

If you would provide the report to a customer, then export it and prepare it for the customer. Do you notice any issues?

2

If you would use the report for operational analysis, then export it and work through a sample analysis. Do you notice any issues?

3

If you would use this report to review data periodically (such as checking on the status of a sales order or work order), then use it as needed for a day or two. Do you notice any issues?

Sign Off

When the report has passed all of your tests, it will be ready to be made live. Ultimately, you are as responsible as IT to ensure that your report is accurate prior to us making it live.

Before the report is considered live, we will ask you to approve making it live via email. This will be considered your Sign Off that the report is complete and accurate.

Creative Commons License
End User Testing Guide for Reports by Amanda Thomas is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

An end user testing guide

One of the most interesting (and often frustrating) changes at work since the merger completed has been trying to help the users understand that I am no longer an expert in our systems and processes. It is a complete shift from what we are all used to.

For the past couple months we’ve been working on building reporting with SAP data to replace the reporting that we had before the merger. I’ve learned a LOT about the data and have honed my Google-fu (so much that one of the experienced SAP users recently referred to me as an SAP wizard when I was able to tell him which tables to look to for descriptions of sales document types and billing document types).

The challenge has been testing. Since I used to know the system and processes so well, I used to do the majority of the testing for new reports or when we made system changes. When end users tested system changes, I would give them a checklist of specific things to test for, types of transactions to test with, and types of issues to look out for. When they tested reports, it was mostly about making sure the columns and search criteria they needed were there. But in both cases they didn’t need to test extensively because I’d already worked with the dev team to eliminate every bug I could find (and I am damn proficient at finding bugs, if I do say so myself).

A couple of weeks ago we got to the point where we’d  developed a bunch of reports, most of them were in various stages of testing but a few had been tested by the users, had bug fixes, and been made live. And for two in particular several rounds of fixes had happened after making them live.

I needed to find a quick and relatively hands-off way of getting the users equipped to test reports. It needed to be quick and hands-off because I had ten reports in development with at least as many people (in three time zones) involved in testing, and because I was having a hard time keeping up with the dev team as it was.

After searching for a quick user testing guide with no luck, I decided to make my own. And I decided since this appears to be something that is lacking in general, I thought I’d share.

So without further adieu, here is a quick testing guide for reports (that can be easily tweaked for other testing as well).

UPDATE 13 May 2013: This has received a bit more traffic than I’d anticipated, so I’ve also created a page dedicated to the testing guide. If you have comments, recommendations, questions, etc, please post them on that page. Any future revisions will be made there.


Why do I need to test?

Testing is essential to making sure that any report or system enhancement is accurate and works as expected. The IT team tests as thoroughly as possible, however there are several reasons that testing should be done by the end user as well:

  1. You are much more familiar with your piece of the system than IT is. SAP is a new system for all of us. While you have been learning how your processes function in SAP, we have been providing cutover support, learning the data structures, and learning how to look up limited pieces of the data.

  2. YOU are the expert in your area of the business. The IT team has limited familiarity with how you use the system. We are constantly learning where various data elements are and how things interact, but we don’t yet know enough about the operations of any business area to consider ourselves experts. YOU are the expert.

  3. Customer satisfaction is important, and YOU are our customer. You are our customer, and it is important to us that the solutions that we provide meet your expectations and needs.

Types of Testing

Testing can vary quite a bit in complexity depending upon the project and the stage of development. There are three essential types of testing for reports:

  1. Interface Review: Validating that the search criteria and columns included in the report are correct and working as expected.

  2. Data Validation: Detailed review of the data presented in the report in comparison to the source system (SAP) to identify any fields that may be incorrect.

  3. Usability Testing: Testing the report as you would normally use it. This testing can help uncover issues in presentation or in the data included in the report that were overlooked in the Interface review and Data Validation.

Identifying Errors

If any problems are found encountered for any of the tests, they need to be documented and sent to the IT team for review.

For problems encountered during Interface Review:

  1. The exact search criteria used
  2. If you received an error: what that error was and instructions for how to reproduce the error. (Screenshots are always welcome!)
  3. If there was a problem with the results, details about how the results differed from expectation.

For problems encountered during Data Validation:

  1. Example transaction
  2. Which column is incorrect
  3. The incorrect data shown
  4. What the correct data should actually be
  5. Where to see the correct data in SAP.

For problems encountered during Usability Testing:

  1. If a data issue is found, then provide the information listed under Data Validation.
  2. If another issue is found, then outline what the issue is and the requested change.

Report Testing Checklist

Interface Review:

#

Test

Pass/Fail

Problems Encountered

1

Are all required columns present?

2

Are all column names correct?

3

Is the sort order for the results correct?

4

Are all required search criteria present?

5

Are all search criteria names correct?

6

Are default search criteria appropriate?

7

Do all search criteria work properly?

8

Are there any other issues or changes needed?

Data Validation:

The following apply to EVERY column.

#

Test

Pass/Fail

Problems Encountered

1

Is the data format correct?

2

Is there data in every column that should contain data?

3

Validate the data shown on the report: If the report contains multiple types of transaction (for example: notification types or billing document types), check at least two of each. Compare the report to what is shown on the transaction(s) in SAP.

4

Validate calculations: If the report contains any calculations (subtotal, days past due, etc), validate that against the other data on the report and against data from SAP.

5

If this report replaces an existing report or data pull that you use, then compare results between the two reports. Do you notice any issues?

6

Are there any other issues or changes needed?

Usability Testing:

Consider how you would use the report in the normal course of business, then test the report as if it were considered live.

Warning: Do not share results from the report or base decisions upon the report unless you have completed Data Validation testing without any issues.

#

Test

Pass/Fail

Problems Encountered

1

If you would provide the report to a customer, then export it and prepare it for the customer. Do you notice any issues?

2

If you would use the report for operational analysis, then export it and work through a sample analysis. Do you notice any issues?

3

If you would use this report to review data periodically (such as checking on the status of a sales order or work order), then use it as needed for a day or two. Do you notice any issues?

Sign Off

When the report has passed all of your tests, it will be ready to be made live. Ultimately, you are as responsible as IT to ensure that your report is accurate prior to us making it live.

Before the report is considered live, we will ask you to approve making it live via email. This will be considered your Sign Off that the report is complete and accurate.

Creative Commons License
End User Testing Guide for Reports by Amanda Thomas is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

I love change, until I hate it.

I have a strange love of change in my work place. I get bored quickly doing the same type of work repeatedly. The more things change, the more my job remains interesting. This is generally good for me because my job typically has a lot of variety, it all leads to some sort of change, small (like new reports to help improve process) or large (such as new system implementations).

When going into a project, I have always been the first to admit that change can be difficult, but that I am weird and LOVE change. I think my enthusiasm usually sets others at ease, although sometimes it can also mislead others about the complexity of the undertaking.

A year ago, the company I work for was bought by another company. I spent most of the year working on the systems integration project with a team from both companies, including about a third of 2012 traveling to the new headquarters on the east coast. In November, the companies officially merged and the system changes went into effect. It meant an ERP (SAP) change for half of the organization and a CRM (SalesForce.com) change the other half of the organization, among a bunch of other changes.

Throughout the process I did the same type of work that I’ve done for the past five plus years: a mix of Business Systems Analyst (geek-to-business-translation) work and Project Management.  Unfortunately, my BSA work was all focused on the legacy systems, not the systems that we were consolidating to.

Once the go-live data issues settled down and my focus changed to the new system rather than the old, I became horribly overwhelmed.

At the risk of sounding like a corporate dweeb… I have prided myself for years on being a change agent of some sort and as part of that, hand holding others through change, and always having the answers.

Last week, I realized that I was struggling with the change. It was taking me days to get through something that would have taken just a few hours in our old system, purely because I didn’t know the database structure and didn’t know the system well enough to find the data sources without help. I didn’t want to ask for help, because I never need help. (“Never” was definitely an oversimplification.)

It was a huge blow to my ego to realize that I was struggling with the change. And then in the back of my head I heard the advice that I give everyone else: “How are they going to know that you need help if you don’t say something? You won’t learn if you don’t ask.”

I eventually got my head out of my ass and asked for help. Things are still taking me longer than they used to, but that’s part of learning a new system (another piece of advice that I often give others and ignore when it applies to myself).

I’ve come to the conclusion that I’m good at change when I realize that change is happening to me, but when it sneaks up on me or I am not in control in some way I can be really bad at it.

Or perhaps I’ll just blame the SAD, I’m sure that had some influence. (By the way, in case you were curious, the light therapy thing I got is working fantastically.)

So many half written posts…

I’m at the Minneapolis St Paul Airport killing time while I wait for my flight home. It is the best quiet “me” time that I’ve had in weeks. It is unbelievably nice.

I’ve had the itch to start writing here again for a few months, but I’ve been so busy (spent half of the past six months away from home) that even though I’ve started several posts I’ve had a hard time making time to finish them.

It looks like I’ll actually have time to write again soon. So now I’m contemplating whether to start with the back story (what has happened in the past year) or to start with what is going on now…

Pen and Paper

I use To Do lists at work all the time. I have a list of every project or issue that I’m working on. My work To Do list is a combination of a list of projects that are in progress and actual tasks that I need to do. It can be overwhelming at times, but it helps me remain focused.

Throughout the week I check off tasks that are completed, update my notes on things that are delayed and add new items. Every Monday morning I review the previous week and create a new list. I may have to set things aside to deal with critical issues or new projects, but at least I’m organized when I am ready to pick up from where I left off.

I used to be organized and productive at home too, but when I returned to school in 2005, my approach to home life changed. When I wasn’t working or studying, I did just what was needed to get by; cooking, laundry, etc. I’ve been officially done with school for just over a month and had expected that by now I would have gotten rid of the clutter in my living room and kitchen, set up a little studio space somewhere in my house for when I feel like painting or being crafty, begun work on a painting I abandoned two years ago, and finished two books that I have half read.

Sure, the holidays happened (and for my family the holidays didn’t end until mid-January), but I’ve also been playing games until I’m beyond bored and watching shows on Netflix when I actually feel like being active. Heck, it has even been two weeks since I my last post (It’s a Post a Week challenge, not a Post When Ever You Get Around To It challenge).

It has become apparent to me that I really need to break the bad habits I formed while in school and become more disciplined. Now this isn’t to say that I’ve been ignoring my New Years resolutions, but I know I can do better.

I know. I’m rambling again.

What I’m getting to is that the way I approach things at work is completely different from how I approach them at home. Thinking through all of this helped me to realize that one simple thing can help me gain some momentum: A To Do list.

My approach will be similar to what I do at work. I’m listing my Resolutions (projects), along with the short term tasks that need to happen (Such as cleaning and rearranging/reorganizing the living room), and any other things that are going on that week (such as: games with friends Saturday).

Also, like at work, I’ll be preparing my To Do list in a pen and paper format. I know there are advantages to making a digital To Do list, but I’ve found that making my To Do list with using pen and paper helps me remember my list and helps me avoid the distraction that my computer can offer (look, StumpleUpon! ooh, shiny!).

Of course, it also takes some discipline to actually do what is on my list, but I don’t think that will be much of a problem.

Would you like one lump, or two?

I haven’t made time to write since last Sunday. So, to keep myself in the habit (and help me sleep better), I’ve decided to share this (relatively unedited train of thought) post. Think of it as the two of us sitting down over tea, and me monopolizing the conversation.

This last week flew by. Work was chaotic (in a verging on maddening yet semi-controlled sort of way), so much that I postponed a day off just so I could take that day knowing that I wasn’t perpetuating my backlog of work.

I mentioned recently that writing here has been helping me sleep better. This past week echoed that reality. I’ve gone back to waking up in the middle of the night thinking about everything and nothing, and having some really strange dreams. I’m sure it is fueled by my excitement for PAX (and then graduation), my eagerness to have a change in my career, and my nervousness about making it happen.

PAX is less than twelve days away now! I’m still fretting about being prepared. My hubby and a couple of our friends have planned what they’re going to wear each day. And they’re guys. I’m trying to get on that bandwagon, but I only have one shirt planned. For the other two, I’ll probably just pick whatever seems appropriate the day before.

I’ve started planning which panels to attend. I’d like to go to all of them, and then have another 3 or 4 days to wander and gawk over/play with all of the games and toys that will be there. The reality is that I’ll probably do a couple of career focused (not fanboy focused) panels per day, and spend the rest of the time exploring the glory that is PAX. Oh, and drinking caffeine. Lots and lots of caffeine.

I did decide to bring “introduction cards” (business cards, but for me – not for any business), because a little help to my networking skills couldn’t hurt. I haven’t decided what to put on them yet though. That may be my next task.

Well, that and completing two weeks worth of school work in the next 10 days.

~ Amanda

A few PAX related links

What is YOUR dream?

I started writing here in November of last year. Ten months ago. I have made 38 posts. Thirty-eight posts in ten months is less than what I had hoped to do when I started, but now that I have been writing here for a while I am getting more comfortable with a routine of writing.

I continue to write here both as an outlet. I wouldn’t call it a creative outlet, because I don’t feel creative at all in what I’ve written here. It is more of an outlet for my hopefulness and anxieties about the dream job and for the many other thoughts that bounce around in my head at the end of the day (or even during the day; such as when I post about frustrations at work). It is actually helping me sleep better.

I also continue to write here because it is a constant motivator toward the dream. Each post seems to make me more resolute that I will achieve the dream.

Recently, I’ve felt a little self-centered in all of this dreaming. I write about what I want to be when I grow up, or the different fun things that I’m doing to (hopefully) help me get there. I talk about it with friends over drinks. But I don’t ask anyone else what they want.

I am going to change that. I am starting here. (If you just got Man in the Mirror stuck in your head, you’re not alone.)

What is your dream?
What are you doing to work toward it?
Do you have a plan?
What motivates you?

I know it’s weird that some random internet person is asking, but it’s amazing the motivation that can stem from kind comments from someone you barely know and from knowing the dreams of others.

Still looking for that reset button.

Today is already getting better.

Earlier today I tweeted about my day being off to a bad start. It was. I woke up feeling hung over and aching all over (even though last night I only drank water). I had one co-worker upset about a change request I’d sent them after testing an update to one of our internal systems. I’m sure I annoyed the heck out of another with a few emails for little changes on a different project (fortunately, the second co-worker didn’t complain).

(more…)

Preparing for PAX

I get to go to PAX this year! One month from now, the gamer masses will converge on Seattle for a three days of geekyness (as will music lovers for a different event, expect traffic). This will be my first PAX.

For the past week I haven’t been able to get PAX out of my head. Going to PAX is a big deal for me, both as a fangirl and as someone who wants to work in the video game industry. My inner project manager is planning what to bring and what to do, and making lists. A good plan starts with a list of things that need to happen right? (Well, after an objective, but that is already defined: PAX + Fun + Networking.) 

(more…)