Adzerk Team Blog

This post is about our foray into the marketing automation jungle. As a SaaS start-up our budget is meager and our resources limited but our resolve is mighty! I’ll talk a bit about what tools we’re using, how we’ve got it setup, what we love and hate about it and what we know we need to improve on.

Early on we decided to use Salesforce as our CRM solution so this was technically the first piece of our marketing automation puzzle. We were tracking leads and opportunities in Salesforce but not much more than that. New trial customers would get added to Salesforce and we would work our butts off to turn them into paying customers. The leads that didn’t turn into customers were left in a state of limbo and every once in a while we would send out a newsletter to all the email addresses we had acquired. 

Our marketing efforts were in their infancy so most of our sales activity was outbound with very little in-bound. We had big plans though…

The RunofNetwork blog was started to supply a source of advertising information for publishers and networks. We hoped RunofNetwork readers would explore Adzerk after engaging with the great content provided by the blog. We also launch PPC campaigns to target prospects searching for relevant terms. Articles in TechCrunch and AdExchanger helped to get the word out about Adzerk as well. And of course, there’s this Team Blog where were introduced Adzerk to the non-advertising world.

Leads started coming in. Soon sales was busy just fielding leads and didn’t have much time for cold calling. We had a new problem: Not all inbound leads were the same value. My Mom signing up for Adzerk because she was proud of her boy was clearly not worth a call from the sales team. How do we weed out the valuable leads from the merely curious readers of an article? How do we nurture a potentially valuable lead that simply isn’t ready for a sales call, but could be a customer in a few months?

We could hire some sales people to handle the growing inbound leads and send regularly scheduled followup emails. We could probably scale up pretty good with a couple of junior sales guys that did a lot of copy/pasting and escalated to our “enterprise” sales guys when a lead was ready to close. We would probably need a manager to handle the sales team sooner rather than later too… this option would get out of hand quickly in terms of cost and infrastructure for all of the new headcount.

Marketing Automation is the better answer. 

We needed a system that would track our touch points with users and tell us if a user came in because they clicked a link in TechCrunch or searched Google for “I want to switch ad servers today.” This system should also be able to periodically touch base with leads and care for them until they were ripe enough for the sales team to reach out to.

We explored the usual players and quickly narrowed our choices down to Loopfuse and Marketo. Loopfuse was attractive because they had an entry level solution that was free to get started (remember we had a modest budget) and Marketo popular with some of our advisors. Marketo was pretty rigid with its terms and clearly didn’t need a small customer like us so after a call with each we quickly chose Loopfuse because they were easy to work with and eager to have us as a customer (Loopfuse is startup-y that way). 

We plastered Loopfuse pixels everywhere — our website, our landing pages, our signup forms, etc. We even print them out and give them to people that come by our office (ok, that might not be true). We imported our contact lists into Loopfuse and installed the Salesforce app so our sales team has visibility into each leads history and interaction with us (this is one of our favorite features).

I setup a couple of basic Lead flow tracks so we can followup with trial customers and keep tabs leads that come in through the sales form. As a nice perk we were also able to configure Loopfuse to send the entire company an email when a lead signed up or converted to a paying customer. (It helps make everyone feel involved in our sales & marketing success)

I setup some rules that would assign visitors a “score” when they triggered certain events. For example, visiting our home page is worth 1 point but visiting a page about how to start an ad network is worth 20 points and serving an ad gets you some good points too. Loopfuse will periodically check the scores for our leads and if they reach a certain threshold it will notify the sales team and set a lead status in Salesforce to ‘PreQualified’.

The Sales team loves the fact that leads are now segmented so they can focus on PreQualified leads. They are of course able to see all leads and if they have the bandwidth they call even the ones that aren’t pre-qualified — but the focus on valuable leads is a good thing. 

Where do we take this next? 

We need to be better about segmenting our leads and sending out content to stale leads on a regular basis. Loopfuse will make it easy for us to send content that’s crafted to fit each lead, we just need to create the content. 

Lead scoring is an iterative process that also needs attention. Sales and marketing meet often to talk about leads and how we score them better. We’re constantly tweaking our scoring rules so that only truly pre-qualified leads get to sales. An example: Initially we gave a few points whenever you would open one of our emails, until we quickly realized that opening a few emails would give you enough points to be pre-qualified. Sales wasn’t happy to get pre-qualified leads who had great scores just because they opened an email 10 times!

Marketing automation has been great for our business and Loopfuse has been a great tool to get us started, however, I’m eager to see Loopfuse features ‘mature’ a bit more. Their lead flow process can be cumbersome, custom event tracking isn’t quite usable and reporting needs some work. We’re still maturing ourselves though, so I’m ok with a few warts since their product is dependable, supported well and flexible.

The more I spend time on marketing automation and the simple task of supplying valuable leads to our sales team, the more I realize this game requires a lot of content. Whether its thank you pages, congratulations emails, check in emails, follow up emails, monthly updates, status updates, discount offers, educational blog posts or even blog posts about our marketing automation processes — content is more than a full time job.

If you read Hacker News or keep up with technologists, you’re probably already aware of how broken the recruiting industry is. Specifically, recruiting for small to medium sized tech companies is so awful that it’s becoming laughable now. 

See, there’s a problem. The need for talented engineers and designers is at an all time high, and the supply of the unemployed in the industry is lower than normal. It’s a good time in history to know how to create things with a computer. So this inevitably leaves some companies little choice but to hire recruiters to search for manpower since the business doesn’t want to devote their own resources.

A new industry is born! There’s a need for a product (the potential employee), and there are businesses that would love to pay for the product. In far too many cases, a corporation creates boring enterprise Java software, they pay well, but they can’t find enough developers. They hire a recruiting firm to search the interwebs and offer them 5-10% of the salary if the potential employee is hired. A new university graduate gets the entry level recruiting position with hopes of making it big off of the promise of commission. After taking the time to create 10 personalized messages to prospective matches, he’s rejected by all 10. He uses the second half of his day but ultimately fails. All the engineers are currently happy at their current job, and since he can’t disclose the company, the position, or the project, most reject him or don’t respond. 

So where does that leave the recruiter? It’s been three weeks, and he hasn’t gotten anyone hired and has only sent one engineer to an interview which could possible go awry. It’s time to roll up the sleeves! The recruiter wakes up at 6AM the next morning with an intent of generating a spreadsheet full of leads. The position he’s looking to fill require 5 years of C# and SQL Server experience and also a Bachelor’s degree in Computer Science. LinkedIn is his new favorite tool. All he has to do is enter “C# SQL Server” into the search field. 226,979 results return! If he messages 100 of these results, there are surely a few people to respond. And so it begins. A quick scan over the LinkedIn profile proves to be enough, and he sends the same message he’s sent to others today. 

If you’ve never received these LinkedIn messages, consider yourself very lucky. Here’s one of the two that I received today:

Hi Kacy, 

I came across your profile this morning, and I wanted to reach out and see if you were open to hearing about some new opportunities. Let me know what you think.

Honestly, there was a recruiter a few months ago that wanted me to apply to a Senior Software Engineer position that had a starting salary of $140,000. I was flattered, but there is no way I could be qualified for the position since I only have a few years of industry experience. A quick Google search or further investigation of my LinkedIn profile would have shown that I was not the ideal candidate for the position. 

David Heinemeier Hansson, web famous for creating the popular web framework Ruby on Rails, is well known in internet circles to be outspoken and opinionated. So when technical recruiters pitch him, he occasionally publishes the email so that others can see how clueless recruiters can be. Once, there was a woman that asked how many years of Rails experience he had. He responded with “All of them”. Another example of how a simple 10 second Google search could have saved embarrassment. 

So how do we fix this? Here are a few of my suggestions:

Actually talk about the project. If it’s a contract or a full-time position, I’m not going to leave my current job (which is possibly the coolest in NC) for an anonymous engineering position in Minneapolis. It’s just not going to happen. I, like everyone else, would like a position that’s mentally engaging. If I don’t know anything about it, why would I consider applying?

Money isn’t everything! The fact is, the best talent doesn’t work for the money. If it’s one of the few things you can disclose, put it out there in the beginning. If it’s a startup looking to hire, let me know! Maybe there’s equity that can be worked out. 

Please never call us. It’s just not our thing. Email is far more efficient. A few months ago, a recruiter called the sales line at my current company asking for me. My co-worker told them I wasn’t available, but the recruiter wouldn’t say what it was about. I was worried that something may have been wrong with a family member or there was a financial issue with a bank, but no… it was just a unprofessional recruiter looking for a lead.

Be active in the community! Attend developer meetups or network at job fairs. Don’t spam us with job openings, but be aware of people that could be looking for employment.

Be transparent. Tell me about the position you’re trying to fill. Is there a lot of interest? How long have you been trying to fill it? How much are you making out of the deal? If it’s not a fit for me and you’re open to it, I’ll try to find unemployed or unhappy friends that could possibly fit the role. If it’s a generic email with no personality, I’ve got a keyboard shortcut for archiving email.

Have some extra suggestions? Feel free to join in on the discussion at Hacker News or below in the comments.

Update: It looks like the DHH post was actually a joke by a commenter. 

Go Bold or Go Home in your Sales Communications

When starting at Adzerk, I was basically given the reins to the sales process so I started doing research to find the best ways to handle customers that you are talking with through the sales process.  I have always been told to ‘be brief, be bold and be gone’ in different sales trainings, but I wanted to make sure that point hit home when creating the process from scratch.  I relied on past experiences from different Sales Executives I have worked with along with doing some research on the best ways to accomplish this.  The result: a sales process that has the highest engagement rate that I have seen.

The first couple of months in the early stages of creating the sales process, the interactions went okay, but the response from my communications were minimal at best.  They were your run of the mill sales emails.  Here is an example of one of them:

Hey there,

First off, welcome to Adzerk.  We look forward to working with you and your ad serving needs.  I wanted to introduce myself as your point of contact through your trial account with Adzerk and also go over some suggestions on how to make the most out of your trial with us.

Are you free in the coming days to discuss in more detail?

Short and sweet, yes, but I found myself not getting much from this even when people were responding.  Another big downside to this email is there was no real value to the customer.  Every email you get today says how their product is the latest and greatest.  So that left Adzerk lingering in the inbox with all of the other ‘awesome’ emails you get on a daily basis.  I am not one to stick in with a crowd, so this was just not cutting it for me.  On to the next one…

I had to add value, something that would make the customer say ‘I’m glad I read that, it was helpful.’  One asset I hadn’t taken advantage of yet was our one-of-a-kind blog in the ad operations industry, the Run of Network blog (I would be surprised if you didn’t want to read it after that endorsement).  I decided to start adding it to some of my emails to add value if they aren’t ready to buy or just looking around.  Here is how the new email looked:

Hey there,

I wanted to touch base with you now that it has been about a week since you signed up for your trial account to see how things were progressing.  I also wanted to see what you were looking to use Adzerk for so I can point you in the right direction and offer some helpful suggestions on how to get the most out of your trial account with us.

When do you have a few minutes to connect to go over these details?

In the meantime, I think you will find our run of network blog beneficial because we update it regularly with useful ad operation articles.  You can find it here: http://runofnetwork.adzerk.com

Again, short and sweet, but nothing happened as a result of this email.  The customer may have got some valuable information, which I am pumped about, but where did it leave me - no where.  On to the next one…

Finally, I decided to ‘Go Bold or Go Home’ after doing some major research on the topic.  I decided I wanted to come up with bold ways to make the customer chuckle, stand out in the sea of emails and find out what I needed to know about each and every customer.  Do they all respond?  No.  But, I can at least imagine that they got a good laugh out of my communications and I find out pretty quick if they are serious or not.  I stumbled upon a blog post from our friend, Eric Boggs over at Argyle Social that mentioned the exact same points with some examples so I started work on some iterations of what they had.  Here is a look at one of many emails that are brief and bold:

Hey there,

You’ve had your Adzerk account for a while now and I noticed you still haven’t scratched the surface on what Adzerk can do for you.  I’ve reached out before but we haven’t heard from you regarding your use of Adzerk so I’m hoping you can simply tell me which option best describes where you stand:

A. We haven’t made a decision yet, but Adzerk looks freakin’ awesome!  We just need one more reason to pull the trigger.

B. I’ll be serving ads soon, settle down.  it takes time to move ad serving over.

C. You made my day! I serve over 100 million impressions each month or have a unique set of requirements and would like to talk to you about a custom package.

D. Adzerk did not fit my needs - leave me alone (Note: it would be real helpful to get your feedback on why it was not a fit so we can make sure we don’t waste anyone else’s time) :)

E. I just want to see how long I have to ignore you before you’ll stop sending me emails.

Let me know what option suits you and I can act accordingly.

As you can see, my goal was to make the person reading it laugh and get into a normal conversation with me.  No need to make them feel tense about a cool product right?  My thoughts exactly.  It actually serves other purposes for us as well.  One, it gives the customer an easy response to the email.  All they have to do is put A, B, C, D or E in a reply email to me.  It also allows us to see where the customer is at in their buying cycle.  Don’t decide for them, let them tell you.  If they just send an email back that says B for example, I know to back off and let them do their thing.  This folks is what I like to call a win-win solution.

This email, along with other iterations have become a big part of the sales process at Adzerk and our customers love getting them now.  Like I mentioned previously, it is the highest engagement rate I have personally seen in a sales role.  If you don’t believe me, check out a couple of awesome comments from customers:

I’m waiting to see what happens if I don’t respond ;) the emails get better and better haha

I am just checking what it is about.  It seems to be interesting but I am looking more for places to advertise our product than to show other ads on our websites.  So probably A ;)

Warning: If you do it correctly, you will get a lot of smiley faces in your reply emails and it will make you feel good about yourself.

I am always up for seeing other cool emails that companies use, so If you have any cool sales communications, I would love to see them.  You can either post here or shoot me a quick email ron [at] adzerk [.] com.

We love user data here at Adzerk. We really can’t get enough of it- we’re running at least five different metrics gathering tools in our application and on our marketing site. They all have their strengths and weaknesses, but Crazy Egg is our go-to tool for visualized user information.

What it is & why it’s awesome

Crazy Egg is a hosted service that provides a variety of visual representations of click data. It’s awesome because click data can be time-consuming to analyze. Sometimes we just need to know if a particular button is being used, and we need to know inside of 5 minutes. Having access to this level of information already conveniently organized is extremely useful in our quest to provide an intuitive user experience.

Getting up and running

The initial setup was a bit of a challenge. Crazy Egg is easiest to use for websites that don’t require a user login. adOS is completely hidden behind authentication, in addition to using a unique url for each customer. So, we needed to include the Crazy Egg tracking code in the application, but in such a way that it could still talk to Crazy Egg from behind authentication.

In order to get around this, we followed this post by Patrick McKenzie of Kalzumeus Software, which provides a great hack. We added a bit of black magic to the tracking code which I won’t delve into in the interests of security. (But seriously, please don’t hack us. You wouldn’t like our DevOps guy when he’s angry…)

Since most of our pages use dynamically generated urls, we set up a series of CrazyEgg snapshots using wildcards. for example- to add tracking to our Channels page, the url on my production account is http://sarah.adzerk.net/network/89/channels. It’s as good a pattern as any to point Crazy Egg at, since every single account has a different url.

I set up a wildcard pattern of http://*.adzerk.net/network/*/channels to match Channels pages across all Adzerk customer accounts. The only other thing you have to set is for how long you’d like the snapshot to run. Some I set to be ongoing, but if I want to test something specific, I might set a user limit (say, 1,000 users) or a specific set of dates, if I know we’ll be rolling out changes.



This is cool because you don’t have to go through and aggregate all the data yourself, you just point to a page and it does the rest.

What it can do

Here’s what we can find out from this one snapshot:

Areas of the page where users are clicking

How much users are scrolling

We often use this to determine if the page is too long and users aren’t finding things because of it, or if they’re scrolling because they aren’t seeing the content that’s actually above the fold.

Clicks, by location and by referrer (which is super useful for us, and possibly my favorite feature)

The Confetti map also allows you to display clicks by a variety of other variables, including browser, operating system, various units of time, and custom metrics that you can set up yourself.
Cases where we use this could include, “Hey, is anyone clicking this link?” Crazy Egg: “Yes, Big Customer X is clicking that link”. “Well alrighty then.”

Clicks by percentage

“95% of the users on this page think this graphic is clickable. We should do something about that.”

List of elements on the page, by type and percentage of clicks, both visible and invisible

You can also compare snapshots in the same window

In a nutshell, Crazy Egg can take a lot of the thinking out of analytics, and we love using it. What tools do you use to visualize your user data?

Who cares what your Klout score is, or where you checked in for lunch, or that you are the mayor of Chubby’s Tacos…..  Is it important?  Is it a healthy and productive use of technology?  Or more importantly  is this a productive use of your time?  I am more interested in knowing if you are a good parent, spouse, employer, employee, sibling, neighbor, friend… Are you “wasting” your time productively? 

Don’t get me wrong, advancements in technology have led to significant leaps in productivity.  We have numerous examples that we can point to right here at Adzerk where technology makes us better/faster/smarter at what we do.   Those that do not embrace technology will quickly be left behind.  If you still carry around a paper calendar, you may fall into this category.  And there are other reasons to embrace technology besides just productivity.  At a recent dinner party, one of the moms said that her teenage son did not have a facebook page.  This comment received the reaction that you might expect: “Yea,… right!”.  One quick check on facebook revealed that mom was wrong and indeed her son did have a facebook account and was quite active.  Her fear of technology had impacted her awareness of with whom and how her son was communicating.   It is not quite adapt or die, but it might be close.  

On the flip side, there are certainly those that have taken adoption to the extreme, and probably to the point of losing some of these productivity gains.  If you are on social media at all, you have come across these “Super” adopters.  They check their Klout score on a daily basis and are actually annoyed when they find someone with a higher score,  they check in when they wake up and continue checking in until they close their eyes to go to bed.  Do you really care where your wife’s brother’s girlfriend’s aunt ate lunch today?  The noise of social media can be overwhelming at times.  When 95% of the information coming from a “friend”, colleague or application is irrelevant or uninteresting, it is probably time to turn them off.  The  time spent wading through the noise does not justify the 5% that might be interesting or of some value.  At some point we need to ask who is in control; you or the cool new app.  If you are standing out in the rain just to take a picture of a restaurant where you want to post to your Path, if you are getting ready for a business meeting and you ask your host to hang on for a minute while I check in, or if you break in to a sweat if you don’t have your smart phone in hand, you may not be in control.

At Adzerk, productivity rules.  It is important to create boundaries around when to use and when to put away the gadgets/apps. This applies to work as well as personal life.  Meals, meetings, and heads-down productive work time should be as free from the constant interruption that some of our technology presents.  For you late adopters, maybe 2012 is the year for an electronic calendar.  For those of us who get the shakes, if we start to lose Klout, lets look up from our devices and have a meaningful conversation with the people that are important to us.  

A couple months ago I was sitting in a conference room with a member of the Adzerk team. We were using one of our cell phones on speakerphone talking to an experienced marketer in the ad tech space. We were talking through ways to promote Adzerk in the industry and the topic of writing guest posts for popular industry news sites came up. What the marketer suggested was that he write a detailed post about a hot industry topic and then they put my name on it as if I wrote it. The marketer even pointed me to posts on top sites with high profile CEOs listed as the author that he had written for them.

Now what stunned me wasn’t that this happens - I think we all know that it happens form time to time. What shocked me was that this appears to be standard practice in the ad industry. I would bet that 80% or higher of the articles you read coming out of ad tech companies aren’t written by the person on the byline.

Do you really want your first interaction with a potential customer or partner to be based on deception?

I come from blogging. I started my first blog in 2003, gained a small following, and eventually couldn’t keep up with it as I got too busy with a growing family and a number of growing startups. To me writing and blogging is very important for a company - it connects a company with the community in a real and visceral way. It makes your company and the people behind it real - to post stories and posts that aren’t written by you but claim to be written by you is going against everything we have learned in the last 10 years.

Be Real. Be Authentic. Talk Directly to Your Customers.

This is something you just can’t outsource.

At Adzerk we have three different blogs. We have our team blog which is an “anything goes” blog that team members can use to talk about what we are up to, we have a product/company blog that is for official announcements, product releases, etc. Lastly we have the Run of Network blog which is our thoughts on the industry, and we also pay outside authors to write for Run of Network. There was never a question in my mind that when paying outside authors we should give them full credit for what they wrote. On all of these blogs you can be 100% sure that whoever’s name appears on the post or article wrote that post or article.

-James

Experience design is the art and science of finding out what makes life as easy as possible for your users, and then doing it.

In a more traditional organization, there’s often an entire team who dedicate all of their time to developing comprehensive studies, aggregating user data, conducting surveys, and the like.
 
In the fast-paced world of a startup, you don’t really have the time to do all of that. In fact, there’s a movement afoot that says UX Designers shouldn’t be indulging in such ponderous exercises no matter what sort of organization they’re working with. Gathering as much user data as possible is not the problem; data is always a good thing. The problem lies in the tools and methods that have come to be utilized in integrating the data into the software development workflow: deliverables. It’s easy to get bogged down in a slow and cumbersome cycle of wireframing, surveying, studying, testing, rinse and repeat ad nauseum. In the meantime, you’re not fixing known problems or bringing all this data to bear on the development of new features for one of two reasons. Either you’re moving so slowly that the development team has cut UX out of the loop, or new development is being held up while waiting for designs to be perfected. Either case is no good.
 

Adzerk’s instance of AgileZen

Enter Lean UX

Or, the principles of Agile Development applied to traditional UX design. Lean UX values team collaboration over completeness of design. At Adzerk, as previously mentioned in Andy’s post about AgileZen and Kanban, we’re working with Agile Zen and the principles of Kanban to simplify our development cycle and place emphasis on delivering working software as a team, as quickly and simply as possible. In order to integrate UX Design into this process, we came up with the following steps:
 

Working collaboratively helps the team forge solutions quickly.

Figure out what we’re doing

We create a user story in Agile Zen. This usually happens as the result of a bug or as part of a larger feature request. It always includes at least some team discussion beforehand, often with a whiteboard and some preliminary sketching, during which as a group we come up with the primary use cases and user flows.
 

User Validation

This phase can include quick click tests, surveys, or just a phone call to a customer we know would be interested. The main idea is to present what we’re thinking and get feedback. The feedback we gather is then incorporated into the next phase.

Low-resolution design tools encourage iteration, since there’s little invested in the creation of the artifact itself.

UX Working Phase

This is where the feature is fleshed out and designed. The deliverable is working UI code, although paper prototypes are often created as well. We rarely spend time committing anything to pixels prior to the actual code.

Development

After a feature is developed, it’s deployed to production as soon as it’s committed to GitHub. This is where our nifty feature flag system comes into play.
A feature flag keeps unfinished work from being accessible or messing up the application for users, and also allows us to turn on new features selectively for particular customers without turning them on for everyone. This allows an even tighter feedback loop- in a recent example, we redesigned one of our reporting pages. A few emails and chats before we started provided good information as a baseline, and then 3 days later we flipped the feature flag for those specific clients to see what they thought. This process gives us the ability to let customers be involved almost as closely as members of the Adzerk team, which we really dig, and our customers do too. The short turnaround times also keep us honest, and prevent feature creep.

“Supposing is good, finding out is better”

—Mark Twain

Ongoing Evalution

We have a few tools that we use to keep tabs on how things are going, apart from the world of rapid feature development. We run a series of quick weekly reports to keep the team updated on how things are going. Problem areas are noted and addressed as a part of related features/stories to keep from holding up new development. I’ll post in more detail in the future about the specific metrics tools we use.
This system is constantly evolving, as is the use of Kanban in our general development workflow, but for now, we’re hitting the two key areas of getting feedback from customers into the design conversations on the team.

Comparing Notes

Other startups in the world are facing this problem. How have you solved it in your startup or organization? We’re always looking for good tips and feedback to incorporate into our process. What have you found that works, or doesn’t work?

This article is a review of Adzerk’s implementation of Assistly as our CRM solution.  My goal is to share insights into our needs, goals, and how we’ve utilized the Assistly platform to help us track, monitor, and improve our customer support efforts. 

Why we chose Assistly

For starters, it is cheaper for us based on how big we are and our current customer base.  We also like the idea of supporting other startups (or at least they were before getting bought out by Salesforce).  But mainly we simply needed something we could implement quickly to manage our growing number of customer inquiries.  Quickly implementing a product is something you can’t get from some other similarly priced and easy to use CRMs.

Our goals and requirements as a support department

Most important and above anything else is that we want to make it as easy as possible for customers to communicate with us in ways that are easiest for them.  This means Twitter, Facebook, Email, and Chat.  At first glance, per Assistly’s marketed appeal is they facilitate communication across all channels in a single application.  Not many options out there do this natively, and do it well no less!

Secondly, like many support departments, we needed a tool that allows us to collect all the case data we report on such as the nature of the customer inquiry.  Whether the customer’s main reason for reaching out is for information or “how to” questions, reporting a problem/bug, or giving general feedback, Assistly lets us do this through Labels and case specific Custom Fields.  We track top level Labels like the ones mentioned but also specific labels for what part of the application they’re referring to. 

Lastly we need the ability to report on the data that we track in the most effective way possible.  Unfortunately there aren’t many applications that can contain data and manipulate it as well as an excel pivot table but Assistly does a pretty decent job of showing you some top high level metrics in their reporting/analytics tools.  However, it is easier for us to keep reporting metrics in excel to manipulate it easier.  Salesforce blows most competition out of the water in terms of generating reports based on all specific case fields and customer data.  I hope with the new relationship with Salesforce brings many great features and abilities in terms of reporting.  For now, manual processing is working fine (but as we grow I don’t expect it to be sustainable).

How we use the app to generate our own reports

In some ways Assistly’s reporting is even beyond our needs at this point.  We’re a small department and are not really concerned with some of the default reporting metrics built in such as the Time to First Resolution of a case (a built in case status) or anything to do with a case priority (a priority number determined by rules you set for various factors like which channel the case came from or the number of followers a twitter account has that a tweet mentioning @Adzerk has).

What we do is utilize case filters to choose the criteria of cases that we want to view.  We choose the hour range to filter by for any new cases that exclude labels such as spam or test (if we receive spam emails or are testing something) and this gives us our weekly case volume.  We then manually go through it and document the following items in an excel document:

• Account Name or Trial Customer (listed in the case subject for easy view so we don’t have to open the case to see who it was)
• Email, Chat, or Phone call (done with labels instead of the built in channel feature)
• Top Level Labels
  • Info/How To
  • Problem
  • First 30 Days Inquiry
  • Feature Request
  • Feedback
• Feature Specific Labels
• Number of Customer Cases
• Number of Trial Cases
• Number of Trial Customers Spoken with 

From all of this data tracked weekly we can calculate stats such as:

• Average Cases Per Week (Customers & Trial Customers)
• Case Per Customers (Customers Only - # of customers that contacted us versus how many we have that week)
• View trends using line graphs for all of the above.

What we actively monitor

Using case specific custom fields and an external excel document we look at each case as we close it and think about what would have solved this case before the customer had to ask for help.  This includes things like a help/how to document, a change in the application interface, or a new feature.  We document these things in our excel document, give it an ID, and add that ID to the case’s coordinating custom field.  This way we can actively see in excel how many cases are counting towards existing or new topics, which helps us prioritize issues we need to address first.

Overall opinion of Assistly as a CRM

The Assistly CRM solution is pretty snazy and by all means worth a test run to see if it can help you with your needs!  While we don’t use all the features and use others differently than how they are intended, I can definitely see how this is a more cost effective solution if you’re a much larger company and need to manage all the different channels in which customers talk to and about you.  You may not get the robust reporting tools you need right out of the application itself but there are plenty of ways to work around some of these limitations and still get to the bottom line of helping your customers.  

Overall opinion of Assistly as a company to align yourself with

These guys practice what they preach!  Their responsiveness to your questions is great, they are open to feedback, they are transparent with issues that come up (check out this post).

If you haven’t checked them out or given them a test run, you are really missing out on a solution that could potentially be perfect for you.

“A system of local optimums is not an optimum system at all.” - Dr. E. Goldratt

DevOps, the mutant offspring of software development and the subsequent operations to keep that software functioning, is typically an abberaration for most organizations. Most engineering groups create a cultural wall between the developers and the people tasked with installing and running the software. Instances where the disciplines are considered to be a unified skillset are few and far between. Looking around the landscape of technology companies I am happy to see that this artificial division is eroding.

At Adzerk there is no separation between development and operations. All engineers are expected to develop and master skills of writing code and keeping the business operations functioning. 

IS THIS YOU?

If you find yourself squirming uncomfortably at the idea of developers deploying code to production environments, you probably work in an environment that discourages mixing of dev and ops roles. Developers write the code. Operations engineers install the code.

Perhaps there’s a hand off, with a lot or maybe little ceremony. Perhaps you have coordinated release reviews where the heads of departments show up and grunt and nod and sign off on a release. Or maybe you also have a long verification process filled with checklists that takes days or weeks to finish. There’s probably also volumes of expected documentation which acts as a contract designed to insulate your team from blame.

Worse still, if there is a problem with the deployed code there is a frustrating delay to fix it. Developers can’t work on the systems to debug the problem easily, or at all. Production staff insist on overly complicated rituals before they will expose themselves to yet another risky release of code. The feedback cycle on the software lengthens further still, delaying the delivery of actual value from the software. 

Because the production staff is so overwhelmed with rising costs of complexity and risk of each deployment, they demand larger work buffers. In this case the buffers are carefully documented processes and ever more rigorous manual inspection of the product before they deploy. Since they cannot directly control the process of putting quality into the software, they react the only way they can - they must hire more people to make the workload manageable.

Meanwhile, the developers react to the demands of the production staff by grumbling about how production is “dragging their feet” on deploying fixes. Any problems in the deployment is clearly the fault of people who just don’t understand how to work with the product. They are unable to rapidly adjust the product to a changing marketplace. They lengthen their work buffers by creating longer cycles of “requirements gathering” and architecting features no one asked for.

The end result is that the business become slower to respond to change. Development costs begin to climb as projects become delayed. Engineers become frustrated and angry with the workplace. Attrition becomes a problem as the most talented people seek greener pastures. 

Everyone sees problems but no answers. Everyone complains “We can’t get anything done!”

KANBAN

As bad as this sounds, there are ways to turn it around. As Andy explained in a previous post, Kanban is a powerful tool for visualizing and inspecting the flow of software delivery. By creating large and visible representations of how we work, we were able to avoid the trap of dividing our engineering team into functional silos. Rather than focus on efficiency of each work unit (code created per developer) we chose to focus on the pace of the working, software delivered.

Eliminating the “traditional” specialization between development and production is an outcome of our Kanban adoption. Kanban is not the only way to address this problem, of course. Any framework that encourages inspection of how work is done and encouragement of explicit conversations about how to improve that work are necessary conditions.

INFRASTRUCTURE AS CODE

As a corollary to devops as an organizing principle, we’ve adopted “Infrastructure as Code”. This says that the configuration and provisioning of our infrastructure is described and controlled by programming tools. We chose Chef, but there are many tools available such as Puppet and cfengine. Every developer has access to the infrastructure code and the server infrastructure itself. 

If the thought of easy access to the servers that run your business makes you uneasy, it just means you are sufficiently paranoid. We use that unease to develop our version of “poka-yoke”, or error proofing mechanisms. Since we are responsible for the availability of the production environment, we make engineering choices to minimize fear.

CONTINUOUS DELIVERY

Our software is pulled from our Github source code control repositories into a continuous integration pipeline. We apply a battery of fast unit tests. If those tests pass, we immediately and automatically deploy the code to a test environment. The pipeline runs another round of API tests followed by our slowest functional tests. The pipeline shuts down when it detects a failure and notifies everyone.

If the tests pass then the code is promoted to a release. The pipeline then deploys the promoted code automatically to our production servers. 

EMERGENT PROPERTIES

Do we make mistakes? Of course! What we have found is that the longer the time a developer takes to commit changes, the bigger the changeset. And the more that changes, the greater the chance that a bug has been introduced. By forcing all changes to be applied as soon as possible we encourage small changes. 

Small, incremental changes are also easy to debug. If a developer makes a mistake in 3 lines of code, we know we only have to search 3 lines of code for the problem. If your release cycle takes a month, how many lines of code did you change? How many lines of code do you have to inspect to find and fix the defect? Thousands? Tens of thousands? Millions?

The other advantage to making frequent releases is that the time to fix a problem is usually about 15 minutes. We don’t petition a committee to make a change, or fill out a form, or create a ticket proposing the change. We make the change is made and it’s deployed immediately. Our build and deployment infrastructure creates and reports the information we need to audit changes and fix problems.

We think the ability to react this rapidly is an immense advantage. This stance forces everyone to have explicit conversations about what constitutes value to the business. We measure the impact of changes almost immediately, rather than in weeks or months. This culture creates an engineering culture of achievement and pride instead of fear and frustration.

We learn rapidly from our work. We serve our customers better. Win!

We’ve been using Kanban for a couple of months now at Adzerk so I wanted to give you an update on how’s its been going.  spoiler alert: it’s great!

The Way We Were

As mentioned in previous posts, we’ve always used AgileZen to manage our backlog and current development work.  When it was just James, Kacy and me we could operate without too many rules/conventions. Mostly this meant that I snuck a ticket into the queue when James wasn’t looking and it was ignored in favor of shinier objects (don’t tell him I said that).

There were hundreds of tickets in the AgileZen backlog and we would meet regularly to pick which ones to work on next. Often a high priority would come up and get put directly into the working queue alongside existing tickets. It wasn’t uncommon for the queues to get fairly large until we had time to clean them up. Completed items would get moved to the ‘QA’ queue which would backup until we were ready for a deploy and then we would scramble to test everything before releasing into the wild. Unless you wanted to have some sort of roadmap, allow engineers to be laser focused on a small set of tickets and limit a bug’s impact on our customer base then it was all a glorious process that never failed.

A New Age

With the addition of Sarah and Jeffers (not to mention Andrew, Ron, Brian and Steve) we needed a bit more order to ensure we were focusing our resources on the right priorities.  In addition to better organizing our engineering team, we also needed a better way to give timelines to customers and market facing teams.  James gathered us all into a conference room and the wild wild west became the renaissance. 

AgileZen makes it easy to add Kanban limits to each step in the development process so we added limits to the number of tickets each queue could have. We moved forward with the following queues (and limits):

• Ready (7)
• UX (2)
• Working (4)
• Acceptance (4)

Only 7 tickets can be in the Ready queue and we limit the Acceptance queue to 4. The UX and Working queue limits are equal to the number of engineers+1. Somewhat arbitrary limits but there was also some thought put into them.

The key is to obey the Kanban limits and if the Kanban board is clogged you don’t want to continue moving tickets forward, which would be a Kanban violation. The first rule of Kanban is to follow Kanban, so when we catch someone violating the Kanban process by overloading a queue we play this ALERT throughout the office. 

Effects of Kanban

From a product owner’s perspective this is all great because by instituting Kanban limits on each step the product owner has more control over what engineers focus on (he/she controls what goes into the ready queue). As the product owner I use Excel to manage the backlog of bugs/features and keep it prioritized at all times. When there’s room in the Ready queue I create a ticket and add it to AgileZen. When tickets are put into the Acceptance queue it’s the product owner’s responsibility to give final approval that a ticket is complete and works as intended. I get the last look at making sure it’s ready for customers to use and am responsible for clearing tickets off the board when they are complete. 

Kanban encourages a team mentality because if any of the queues reach their limit the whole process gets clogged up which forces everyone to focus on fixing the clog. If there are already four tickets in the acceptance queue then no more tickets can get completed and moved into acceptance until the product owner clears out the queue. If there’s a problem testing a ticket because of the QA environment then it hurts everyone because the Kanban board gets backed up. 

We’re constantly discussing the process and tweaking it to work better for us. If a ticket needs more work should we move it back to the working queue? We decided the answer is NO, tickets never move backwards (but I can see it working both ways). Do we need to have a UX queue? I’ll bet at some point we try getting rid of the UX queue and just have a Working queue with 5 or 6 slots. One new piece of the process for us was adding size attributes to each ticket so we can start tracking velocity. I’m excited about the new insights this will give us but I also expect it will also lead to some tweaks with our processes.  

These were some big changes to adopt. And there was definitely some concern voiced when the Wild Wild West ended. “Rules!? We don’t need no stinking rules!!” The bottom line though is to not let the process get in the way of being productive. I think we’ve succeeded with that and everyone is much happier since the change.  

TEASER

The best part about all of this, and I think everyone’s favorite part, is the use of feature flags. By using feature flags we’re able to continually deploy and keep new code hidden from production accounts. I’ll leave this topic for a future post but it’s a beautiful thing when combined with our Kanban process.