I speak professionally on the topic of mental illness, and almost every time I give a talk about living with a mental illness, at least one person in the audience tells me about a loved who went off his or her psych meds or won’t even start taking them. As we have seen historically, when people living with a mental illness—those who need to take medication for it—go off their meds, things can go sideways. Or, if someone never starts taking them, he or she can have a hard time finding stability.

When I talk with those loved ones of the mentally ill, I can often sense their anger, frustration, and judgment. What I don’t think they realize is how hard it is to be and stay on meds. I should know: I have taken medication for bipolar disorder for 24 years.

The most common reason I hear people deciding to stop taking their medication is the side effects. And although side effects are tough, they aren’t the only reason people go off meds.

Here are four more reasons why the mentally ill may stop taking their medication that people don’t usually talk about (but should):

1: It’s really, really hard to remember to take pills every day.

It’s such a simple concept, but so true! It is so hard to remember to take medications day in and day out. Not to mention that many medications are time-specific, which can make taking them difficult if you’re out and about or, say, in the middle of a meeting. Taking daily medication is more than just creating a new habit; it has to become part of your lifestyle. When you’re at home with a set schedule, it’s not so bad. But, when traveling or being away from home for any amount of time, you have to remember to bring your pills. It’s a lot to think about and is much harder than people realize. There’s a need for greater understanding of human nature, and about the time it takes to make integrating a new habit like taking a medication into daily living.

2: If you take meds, you must admit (and are reminded every day) that you have a mental illness.

Mental illness is portrayed negatively in society, so it’s understandably hard to accept it when you’ve been diagnosed with one. It can take years to fully accept that you have a mental illness. Also, it’s one thing to be diagnosed, but taking it a step further and treating the illness actually makes it real. If you were recently diagnosed and you start taking meds and they work, it can confirm that you need them and have a mental illness. And if you have a mental illness, you will live with the label for the rest of your life. It’s sort of like, if A equals B, and B equals C, then A equals C. Never thought I’d use that formula in real life! I believe that in most cases, this need to accept the label is an unconscious thing that discourages people from taking meds.

3: You have to have insurance to afford medication.

The cost of health care in our country is one of the biggest problems we face as a society. It’s common knowledge that prescription drugs costs have skyrocketed. Even if you have insurance, meds are expensive. One of my medications is $50 a month with insurance—just one of my medications. And I have good insurance. I couldn’t even imagine what it costs without insurance, but most out-of-pocket prescription drugs can range from hundreds to thousands of dollars a month.

Getting health insurance is still confusing and expensive. In addition, you need to be seeing a doctor to get a prescription, and that costs money as well. Without getting too political, if you want to help the mentally ill, please improve the health care system. These barriers and hoops make it very difficult for people who really need medication to stay on it.

4: Society has not accepted psych meds and sees taking them as shameful.

I often say that there is more stigma in the treatment of a mental illness than there is in sharing that you have a mental illness. In the U.S., taking psych meds is still looked down upon. People who take medication for mental illness are classified as pill-poppers or lazy, and the derogatory term “big pharma” is used to try and disparage the use of such meds. In contrast, it’s always seen as a positive thing when people say they went off their antidepressants and found a “natural” approach to treat their disease. There is little to no support from society for people like myself who choose to stay on their psych meds.

For example, I take an antidepressant and an antipsychotic drug. If I were to tell that to a prospective future employer, do you think they would want to hire me? No. It makes me look, well, crazy. But it shouldn’t because those meds make me stable and able to excel in my job, which, by the way, I have been at for more than nine years. Society is still largely uneducated when it comes to psych medication, and that has lasting significant effects on those who need to take them to function in a productive, useful way.

I would ask that until you have actually lived with a mental illness and have to take medications to treat it, try not to judge those who do. It’s an incredibly difficult and lonely journey. We need your support and encouragement, not judgment. For those of you who take meds, don’t give up! When you find the right combo and dosage, it is worth it to stay on them.

Written by Maggie Newcomb