As I have been writing letters as part of the tenure and promotion evaluation of assistant professors at other institutions, I have been struck by the great variation in the "tenure code" documents I have been sent to help me write my letters. Typically, these documents outline an institution's or a department's criteria for tenure and promotion, but the level of specificity of these documents varies a lot.
At the vague end of the spectrum are documents that state that the candidate must show evidence for "scholarly achievement" (or words to that effect). This is useless for my purposes, and I just write my evaluation however I think best. In my experience, these documents are the most common type.
At the definitive end of the spectrum are documents that state that the candidate must have x papers every n years in "respected journals" and at least y grants from external funding agencies. In these cases, which I rarely encounter, I don't really understand the point of an external letter. If the candidate met the criteria and is going to get tenure, why ask for anyone's opinion? Does it matter if I think the x papers were flawed and uninteresting? If not, then don't waste my time. If it matters, then rewrite the document to include some vague statement about "scholarly achievement".
The definitive tenure criteria likely result in fewer tenure appeals than the vague tenure "criteria", and the vague ones surely result in higher levels of anxiety and uncertainty for tenure-track faculty, but the definitive ones have problems as well. For example, unless there is a way to specify the quality of the "respected journals" (probably by impact factor, for lack of a better measure) and the contribution of the candidate to a multi-author paper and whether it is good or bad to have lots of co-authors (including students), there is little point in specifying the number of papers that must be published in n years unless that really is all that matters.
In that case, these documents might as well just say: To obtain tenure, a candidate must have their name somewhere in the author list of a paper published in some journal that someone respects.
I am actually ambivalent about these documents, speaking from the point of view of a letter writer. I know they must exist, but there really is no good way to construct them to be useful to the letter writer (or the candidate): too vague is useless, and too specific is strange and leads to more questions (most of which can't be answered).
As a candidate, you have to find out the 'unwritten' information by figuring out the norms of your department and institution. This is typically accomplished by having an effective mentor, by talking to other assistant professors, and by having a candid talk with your department chair and mentor at various stages along the road to the tenure evaluation. You can also attend informational talks given by deans or deanlets, although, as an assistant professor, I found such workshops disturbing because the information in them conflicted with what my department chair had told me about some issues (e.g., the selection of letter writers). Even so, I identified these conflicts of information, talked to the chair about them, he talked to the dean, and all was eventually sorted out.
As a letter writer, I don't know the 'unwritten' information, and I'd rather not guess. All I can do is try to write a fair letter based on what is in the record and on my perception of the quality of the candidate's research. The people at the candidate's institution will have to sort the rest out for themselves, depending on what they think my criteria/standards for tenure are likely to be; i.e., they can discount my letter (if my criteria seem too harsh), take it seriously (if they think my opinion is well supported), or pick out the parts that agree with their own opinion.
Mostly, I just need to finish these letters so I can stop obsessing over them (one in particular is much more difficult to write than I expected), and then I can move on to my next adventure in Professional Service Activities.