One problem is that there are many different interest rates. So we have two questions: What determines the overall level of interest rates, and what determines the spreads between different interest rates? The latter in turn we can divide into the question of differences in rates between otherwise similar loans of different lengths (term spreads), differences in rates between otherwise similar loans denominated in different currencies, and all the remaining differences, grouped together under the possibly misleading name risk spreads.
In any case, economic theory offers various answers:
1. The orthodox answer, going back to the 18th century, is that the interest rate is a price that equates the desire to save with the desire to borrow. As reformulated in the later 19th century by Bohm-Bawerk, Cassel, etc., that means: The interest rate is the price of goods today relative to goods tomorrow. The interest rate is the price that balances the gains from deferring consumption with our willingness to do so. People generally prefer consumption today to consumption in the future, and because it will be possible to produce more in the future than today, so the interest rate is (normally) positive. This is a theory of all transactions that exchange spending in one period for spending (or income) in another, not specifically a theory of the interest rate on loans.
The Wicksell variant of this, which is today's central-bank orthodoxy, is that there is a well-defined natural interest rate in this sense but that for some reason markets get this one price wrong.
2. An equally old idea is that the interest rate is the price of money. In Hume's writings on money and interest, for instance, he vacillates between this and the previous story. It's not a popular view in the economics profession but it's well-represented in the business world and among populists and monetary reformers,. In this view, money is just another input to the production process, and the interest rate is its price. A creditor, in this view, isn't someone deferring consumption to the future, but someone who -- like a landlord -- receives an income thanks to control of a necessary component of the production process. A business, let's say, that needs to maintain a certain amount of working capital in the form of money or similarly liquid assets, may need to finance it with a loan on which it pays interest. Interest payments are in effect the rental price of money, set by supply and demand like anything else. As I say, this has never been a respectable view in economic theory, but you can find it in more empirical work, like this paper by Gabriel Chodorow-Reich, where credit is described in exactly these terms as an input to current production.
3. Keynes' liquidity-preference story in The General Theory. Here again the interest rate is the price of money. But now instead of asking how much the marginal business borrower will pay for the use of money, we ask how much the marginal wealth owner needs to be compensated to give up the liquidity of money for a less-liquid bond. The other side of the market is given by a fixed stock of bonds; evidently we are dealing with a short enough period that the flow of new borrowing can be ignored, and the bond stock treated as exogenously fixed. With no new borrowing, the link from the interest rate is liked to the real economy because it is used to discount the expected flow of profits from new investment -- not by business owners themselves, but by the stock market. It's an oddly convoluted story.
4. A more general liquidity-preference story. Jorg Bibow, in a couple of his essential articles on the Keynesian theory of liquidity preference, suggests that many of the odd features of the theory are due to Keynes' decision to drop the sophisticated analysis of the financial system from The Treatise on Money and replace it with an assumption of an exogenously fixed money stock. (It's striking that banks play no role in in the General Theory.) But I'm not sure how much simpler this "simplification" actually makes the story, or whether it is even logically coherent; and in any case it's clearly inapplicable to our modern world of bank-created credit money. In principle, it should be possible to tell a more general version of the liquidity preference story, where, instead of wealth holders balancing the income from holding a bond against the liquidity from holding "money," you have banks balancing net income against incremental illiquidity from simultaneously extending a loan and creating a deposit. I'm afraid to say I haven't read the Treatise, so I don't know how much you can find that story there. In any case it doesn't seem to have been developed systematically in later theories of endogenous money, which typically assume that the supply of credit is infinitely elastic except insofar as it's limited by regulation.
5. The interest rate is set by the central bank. This is the orthodox story when we turn to the macro textbook. It's also the story in most heterodox writers. From Wicksell onward, the whole discussion about interest rates in a macroeconomic context is about how the central bank can keep the interest rate at the level that keeps current expenditure at the appropriate level, and what happens if it fails to do so. It is sometimes suggested that the optimal or "natural" interest rate chosen by the central bank should be the the Walrasian intertemporal exchange rate -- explicitly by Hayek, Friedman and sometimes by New Keynesians like Michael Woodford, and more cautiously by Wicksell. But the question of how the central bank sets the interest rate tends to drop out of view. Formally, Woodford has the central bank set the interest rate by giving it a monopoly on lending and borrowing. This hardly describes real economies, of course, but Woodford insists that it doesn't matter since central banks could control the interest rate by standing ready to lend or borrow unlimited amounts at thresholds just above and below their target. The quite different procedures followed by real central banks are irrelevant. 
A variation of this (call it 5a) is where reserve requirements bind and the central bank sets the total quantity of bank credit or money. (In a world of bind reserve requirements, these will be equivalent.) In this case, the long rate is set by the demand for credit, given the policy-determined quantity. The interbank rate is then presumably bid up to the minimum spread banks are willing to lend at. In this setting causality runs from long rates to short rates, and short rates don't really matter.
6. The interest rate is set by convention. This is Keynes' other theory of the interest rate, also introduced in the General Theory but more fully developed in his 1937 article "Alternative Theories of the Rate of Interest." The idea here is that changes in interest rates imply inverse changes in the price of outstanding bonds. So from the lenders' point of view, the expected return on a loan includes not only the yield (as adjusted for default risk), but also the capital gain or loss that will result if interest rates change while the loan is still on their books. The longer the term of the loan, the larger these capital gains or losses will be. I've discussed this on the blog before and may come back to it in the future, but the essential point is that if people are very confident about the future value of long rates (or at least that they will not fall below some floor) then the current rate cannot get very far from that future expected rate, no matter what short rates are doing, because as the current long rate moves away from the expected long rate expected capital gains come to dominate the current yield. Take the extreme case of a perpetuity where market participants are sure that the rate will be 5% a year from now. Suppose the short rate is initially 5% also, and falls to 0. Then the rate on the perpetuity will fall to just under 4.8% and no lower, because at that rate the nearly 5% spread over the short rate just compensates market participants for the capital loss they expect when long rates return to their normal level. (Obviously, this is not consistent with rational expectations.) These kinds of self-stabilizing conventional expectations are the reason why, as Bibow puts it, "a liquidity trap ... may arise at any level of interest." A liquidity trap is an anti-bubble, if you like.
What do we think about these different stories?
I'm confident that the first story is wrong. There is no useful sense in which the interest rate on debt contracts -- either as set by markets or as target by the central bank -- is the price of goods today in terms of goods tomorrow. The attempt to understand interest rates in terms of the allocation across time of scarce means to alternative ends is a dead end. Some other intellectual baggage that should overboard with the "natural" rate of interest are the "real"rate of interest, the idea of consumption loans, and the intertemporal budget constraint.
But negative criticism of orthodoxy is too easy. The real work is to make a positive case for an alternative. I don't see a satisfactory one here.
The second and third stories depend on the existence of "money" as a distinct asset with a measurable, exogenously fixed quantity. This might be a usable assumption in some historical contexts -- or it might not -- but it clearly does not describe modern financial systems. Woodford is right about that.
The fifth story is clearly right with respect short rates, or at least it was until recently. But it's incomplete. As an empirical matter, it is true that interbank rates and similar short market rates closely follow the policy rate. The question is, why? The usual answer is that the central bank is the monopoly supplier of base money, and base money is used for settlement between banks. This may be so, but it doesn't have to be. Plenty of financial systems have existed without central banks, and banks still managed to make payments to each other somehow. And where central banks exist, they don't always have a monopoly on interbank settlement. During the 19th century, the primary tool of monetary policy at the Bank of England was the discount rate -- the discount off of face value that the bank would pay for eligible securities (usually trade credit). But if the discount rate was too high -- if the bank offered too little cash for securities -- private banks would stop discounting securities at the central bank, and instead find some other bank that was willing to give them cash on more favorable terms. This was the problem of "making bank rate effective," and it was a serious concern for 19th century central banks. If they tried to raise interest rates too high, they would "lose contact with the market" as banks simply went elsewhere for liquidity.
Obviously, this isn't a problem today -- when the Fed last raised policy rates in the mid-2000s, short market rates rose right along with it. Or more dramatically, Brazil's central bank held nominal interest rates around 20 percent for nearly a decade, while inflation averaged around 8 percent.  In cases like these, the central bank evidently is able to keep short rates high by limiting the supply of reserves. But why in that case doesn't the financial system develop private substitutes for reserves? Mervyn King blandly dismisses this question by saying that "it does not matter in principle whether the disequilibrium in the money market is an aggregate net shortage or a net surplus of funds—control of prices or quantities carries across irrespective of whether the central bank is the monopoly supplier or demander of its own liabilities."  Clearly, the central bank cannot be both the monopoly supplier and the monopoly demander of reserves, at least not if it wants to have any effect on the rest of the world. The relevant question -- to which King offers no answer -- is why there are no private substitutes for central bank reserves. Is it simply a matter of legal restrictions on interbank settlements using any other asset? But then why has this one regulatory barrier remained impassable while banks have tunneled through so many others? Anyway, going forward the question may be moot if reserves remain abundant, as they will if the Fed does not shrink its balance sheet back to pre-crisis levels. In that case, new tools will be required to make the policy rate effective.
The sixth story is the one I'm most certain of. First, because it can be stated precisely in terms of asset market equilibrium. Second, because it is consistent with what we see historically. Long term interest rates are quite stable over very long periods. Third, it's consistent with what market participants say: It's easy to find bond market participants saying that some rate is "too low" and won't continue, regardless of what the Fed might think. Last, but not least from my point of view, this view is clearly articulated by Keynes and by Post Keynesians like Bibow. But while I feel sure this is part of the story, it can't be the whole story. First, because even if a conventional level of interest rates is self-stabilizing in the long run, there are clearly forces of supply and demand in credit markets that push long rates away from this level in the short run. This is even more true if what convention sets is less a level of interest rates, than a floor. And second, because Keynes also says clearly that conventions can change, and in particular that a central bank that holds short rates outside the range bond markets consider reasonable for long enough, will be able to change the definition of reasonable. So that brings us back to the question of how it is that central banks are able to set short rates.
I think the fundamental answer lies behind door number 4. I think there should be a way of describing interest rates as the price of liquidity, where liquidity refers to the capacity to honor one's promises, and not just to some particular asset. In this sense, the scarce resource that interest is pricing is trust. And monetary policy then is at root indistinguishable from the lender of last resort function -- both are aspects of the central bank's role of standing in as guarantor for commitments within the financial system. You can find elements of this view in the Keynesian literature, and in earlier writers going back to Thornton 200-plus years ago. But I haven't seen it stated systematically in way that I find satisfactory.
UPDATE: For some reason I brought up the idea of the interest rate as the price of money without mentioning the classic statement of this view by Walter Bagehot. Bagehot uses the term "price of money" or "value of money" interchangeably with "discount rate" as synonyms for the interest rate. The discussion in chapter 5 of Lombard Street is worth quoting at length:
Many persons believe that the Bank of England has some peculiar power of fixing the value of money. They see that the Bank of England varies its minimum rate of discount from time to time, and that, more or less, all other banks follow its lead, and charge much as it charges; and they are puzzled why this should be. 'Money,' as economists teach, 'is a commodity, and only a commodity;' why then, it is asked, is its value fixed in so odd a way, and not the way in which the value of all other commodities is fixed?
There is at bottom, however, no difficulty in the matter. The value of money is settled, like that of all other commodities, by supply and demand... A very considerable holder of an article may, for a time, vitally affect its value if he lay down the minimum price which he will take, and obstinately adhere to it. This is the way in which the value of money in Lombard Street is settled. The Bank of England used to be a predominant, and is still a most important, dealer in money. It lays down the least price at which alone it will dispose of its stock, and this, for the most part, enables other dealers to obtain that price, or something near it. ...
There is, therefore, no ground for believing, as is so common, that the value of money is settled by different causes than those which affect the value of other commodities, or that the Bank of England has any despotism in that matter. It has the power of a large holder of money, and no more. Even formerly, when its monetary powers were greater and its rivals weaker, it had no absolute control. It was simply a large corporate dealer, making bids and much influencing—though in no sense compelling—other dealers thereby.
But though the value of money is not settled in an exceptional way, there is nevertheless a peculiarity about it, as there is about many articles. It is a commodity subject to great fluctuations of value, and those fluctuations are easily produced by a slight excess or a slight deficiency of quantity. Up to a certain point money is a necessity. If a merchant has acceptances to meet to-morrow, money he must and will find today at some price or other. And it is this urgent need of the whole body of merchants which runs up the value of money so wildly and to such a height in a great panic....
If money were all held by the owners of it, or by banks which did not pay an interest for it, the value of money might not fall so fast. ... The possessors would be under no necessity to employ it all; they might employ part at a high rate rather than all at a low rate. But in Lombard Street money is very largely held by those who do pay an interest for it, and such persons must employ it all, or almost all, for they have much to pay out with one hand, and unless they receive much with the other they will be ruined. Such persons do not so much care what is the rate of interest at which they employ their money: they can reduce the interest they pay in proportion to that which they can make. The vital point to them is to employ it at some rate...
The fluctuations in the value of money are therefore greater than those on the value of most other commodities. At times there is an excessive pressure to borrow it, and at times an excessive pressure to lend it, and so the price is forced up and down.The relevant point in this context is the explicit statement that the interest, or discount, rate is set by the supply and demand for money. But there are a couple other noteworthy things. First, the concept of supply and demand is one of monopolistic competition, in which lenders are not price takers, but actively trade off markup against market share. And second, that the demand for money (i.e. credit) is highly inelastic because money is needed not only or mainly to purchase goods and services, but first and foremost to meet contractual money commitments.
 See Perry Mehrling's useful review. Most of the text of Woodford's textbook can be downloaded for free here. The introduction is nontechnical and is fascinating reading if you're interested in this stuff.
 Which is sort of a problem for Noah Smith's neo-Fisherite view.
 in the same speech, King observes that "During the 19th century, the Bank of England devoted considerable attention to making bank rate ‘effective’." His implication is that central banks have always been able to control interest rates. But this is somewhat misleading, from my point of view: the Bank devoted so much attention to making its rate "effective" precisely because of the occasions when it failed to do so.