Like that FiveThirtyEight piece I linked to points out, releasing everyone that's in prison for primarily drug crimes will only reduce the prison population by 14%. The headline figures of 50% that you hear includes anyone that has any drug charge as part of their conviction, even if it's not the most serious charge.
14% is a sizable share, and, in any case, a large portion of those with drug and other charges, rather than just did charges, have all the charges stemming from drug-related activities or transactions, or otherwise connected to drug prohibition (similarly many with no drug charges, but only other charges); ending the war on drugs wouldn't just eliminate the imprisonment tied solely to dig charges.