I might give them the benefit of the doubt and try them again ('t has been years since)

The $ is unquoted and so it’s interpreted as a if variable name will follow. That does not happen (a literal string follows the $) so it casts an error

You probably are looking for this : rm -r ‘folder’$‘003’

The slash will cause the shell or interpret the next character literally (as as $ and not as variable indicator)