How can we tell if we are using a good SEO agency?

A good SEO agency will give you the good news, bad news, and everything up front. They will explain what it is they are doing (not necessarily proprietary information), and will always do things the "white hat" way. What that means is, they follow the rules, and do things that help, and avoid doing "tricks of the trade," which help temporarily, but in the long term can get you booted. I have worked with a company that had been booted due to the efforts of their previous Marketing company, and let me tell you, it's not an easy task to come back from. You could see your website traffic drop over 95%, as well as your sales, overnight. So while your SEO agency might think these tricks are great because they make their clients happy, in the long run, you, the business owner loses, and you'll have an uphill battle that could take years to restore your previous rank.