This paper introduces a framework for evaluating the guardrails of large language models, focusing on Vicuna-13B. We assess its ability to learn to avoid generating harmful responses under 10 red-teaming methods. We provide a dataset with teaching prompts designed to elude the LLM from producing harmful responses, and two additional datasets containing red-teaming prompts. Our findings underscore the effectiveness of diverse teaching techniques in mitigating specific red-teaming impacts.