Whether you pay more in fees going through a bank or a broker depends on which bank or broker you happen to go through.

Let's assume that we are talking about a plain vanilla loan, e.g., the kind given to someone who can document their income, has 2 years on the job, has a 680 credit score or better and who is putting down a 20% down payment. So long as this loan did not exceed the maximum loan amounts set by the government-sponsored entities, i.e., Fannie Mae and Freddie Mac, we would refer to this type of loan as a "conventional/conforming" loan. Such a loan is also referred to as an "A" paper loan.

These loans get sold in groups to either Fannie Mae or Freddie Mac. These organizations, in turn, securitize these pools of loans, using them as collateral for the bonds they issue. The interest on the loans is used to pay interest to the investors who buy the bonds. All of these bonds pay a uniform rate of interest and have a uniform face value. However, when investors buy and sell these bonds, their prices fluctuate in accordance with the effective rate of interest these investors expect to earn on their investments.

Now, getting closer to the answer to your question, let's say that on a given day, investors expect a return of 5.5% on their investment. The price they will pay for a bond giving them that investment return will be priced accordingly. If a bond is offered in the marketplace that pays a higher return than 5.5%, investors will pay more for it. In investment circles, they say that the investor will pay a "premium." Of course, the amount of the premium depends on the amount by which the interest rate of the bond (and the interest rate of the mortgage loans that generate the money) exceeds the market's expectations. So, if the market expects to earn 5.5% on its bonds, and a mortgage bank sells a pool of loans that generates 7% interest, they get paid more for their loans than the amount of money they lent.

Because the mortgage market is very competitive, and as illustrated above, they all get their money from the same place, i.e. Wall Street investors, the rates offered by the likes of Bank of America, Washington Mutual, JP Morgan Chase, etc. are pretty much the same at any given time. It may be somewhat of an oversimplification to say so, but the only reasons that Chase, for example, might have a slightly higher or lower rate or cost structure for its mortgages than Bank of America is because of overhead and profit expectations.

Now, when we bring brokers into the mix, the picture changes a little, but only slightly; basically, though, it is dependent on the interest rate paid by the borrower, as illustrated above. Here's how it works: lenders often have retail and wholesale lending channels. The retail channel, for example would be your local "Acme Bank" branch office. At the same time, Acme Bank might have a subsidiary called Acme Wholesale Mortgage, which enters into relationships with mortgage brokers, who in turn, offer Acme Wholesale Mortgage's loans (and those of other wholesale lenders) to the public. For Acme Bank, the difference is that it costs them much less money to make a loan through their wholesale channel than through their retail channel.

Let's go back to the example of how banks "mark up" the rate of money they borrow on Wall Street through the issuance of bonds to make loans. Let's say Acme Bank has to charge a rate of 7% to one of its retail customers to cover its overhead and make a profit. Because of its lower cost structure, on the other hand Acme Wholesale Mortgage only has to charge 6.25% to cover its overhead and make the same profit. So, they go to the brokers with whom they have relationships and tell them, "if you bring us a loan today for which the borrower is charged a 6.25% rate of interest, we won't pay you a commission. However, to the extent that the borrower pays more than 6.25%, we will give you a certain percentage of the loan amount." In industry parlance, this is called "yield spread premium." The broker could decide to offer its borrowers loans through Acme Wholesale at a rate of 7%, just like Acme Bank, it could charge more, or it could charge less. It all depends on the circumstances.

I've worked at for both brokers and direct lenders. There have been days where, as a broker, I could offer a more competitive rate and fee structure through a bank's wholesale mortgage channel than that very same bank offered through its own retail channel. There have been other days when I could not. The bottom line is that as far as rates and fees are concerned, you have to shop around and do your homework.