Yes, it was either painted or smathered with a thin, soupy layer of something like stucco. That's peeling off, big time, and would likely have to be sand blasted.
<font color="blue"> I am concerned as to why the joints are failing. </font>
I'm not an expert, but it looks to me like there is moisture migrating from the inside out. I'm afraid of heights, so haven't been to the top to look, but I believe there is a concrete cap around the flue that has failed, or, having been through other work done by the original contractor (who was ridden out of town on a rail after having been tarred and feathered, or should have been if he wasn't), it wouldn't surprise me to discover that the concrete blocks are exposed at the top. I also don't know exactly how the flue was constructed -- there are firebricks inside the firebox, but I don't know if the chimney is brick lined all the way up, or if the concrete blocks are exposed. I think it's brick, or possibly clay tile, I know it's not metal.
If it is moisture migrating though the clock and out the joints, it's far from happening at every joint. Most of them are in good shape, and it's only the thin, gruel-like coating (stucco wash or texture paint) that makes it look bad. But, a few joints have had chips of mortar break out of them.
Another indicator is the interior basement wall. It's really more of a full-height, dirt floor, "crawl" space, and the block walls are exposed. It forms the fourth wall of the chimney; the other three are an exterior projection from the foundation wall. The interior of the block wall at the chimney is damp after it rains.
It's been there for over 30 years, and has looked exactly the same in the 16 years I have owned it, so I don't think it's in any danger of falling down.
I had a mason who specializes in decorative stone facings look at it about 10 years ago, and he recommended tearing it down to the firebox and building it back up again before facing it. However, he considered himself an "artist" and wouldn't do the block work. We got involved in some other priorities and let it slide. But, now, the wood shake roof needs to be replaced (it was shingled with siding shingles rather than roof shingles, that dad-blasted contractor, again). We're planning on a metal roof, and would do the chimney work at the same time.
Now that we've got our rural acreage, we no longer have the desire to "get away" from our neighbors, and will likely sell the cabin. I have 3 choices. (1) Do nothing, discount the price for the work, and let the next owner do it the way they want it. (2) Do good repair work, get the increased value (it would be such a dramatic improvement that I should get a much larger return than the cost), and be able to sleep at night, knowing that the repairs will last. (3) Do just enough to make it look good and get the increased value at a lower cost. I would have difficulty doing this. All of the other work I have done on the cabin has been done "right" without much regard to cost. If I do the roof and chimney right, someone would get a very nice little cabin. I'll make a bundle regardless; prices have increased dramatically over 16 years.
I'm not sure if the damper was cast iron or heavy sheet metal; I'm trying to recall what it felt like the last time I reached up in there and manipulated it. The mechanism is broken, but I can still prop the damper plate up in full open mode, or lay it flat on the framework. If it was cast, then it wasn't very thick or heavy. The throat is formed with masonary.
It probably doesn't help, but I just found this picture of the hearth.