Waste less time on Facebook — follow Brilliant.

Please help

Find \(f(x)\) such that \(\displaystyle \int _{ -\infty }^{ +\infty }{ log(f(x))f(x)dx } =S \quad is\quad maximised \)

Given that

  • \(\\ \displaystyle \int _{ -\infty }^{ +\infty }{ f(x)dx } =1\)

  • \(\displaystyle \int _{ -\infty }^{ +\infty }{ { x }^{ 2 } } f(x)dx=k\quad (k\in R)\)

additionally you may use the fact that \(f(x)\) is an even function

Please help if you can, i am unable to solve it

Note by Mvs Saketh
2 years, 4 months ago

No vote yet
1 vote


Sort by:

Top Newest

First of all, I think you want to maximize the differential entropy, i.e. minimize the quantity (not maximize) \[\int_{-\infty}^{\infty}\log(f(x)) f(x) dx \] Subject to the given conditions on \(f(x)\). We start with a tiny little exercise :

Exercise : Show that for any two probability densities \(f(x)\) and \(g(x)\), the following holds\[\int_{-\infty}^{\infty} f(x) \log \frac{f(x)}{g(x)}dx \geq 0 \hspace{10pt}.... (1)\] with equality iff \(f(x)\equiv g(x), \forall x \in \mathbb{R}\).

[Hint : Use the inequality \(\exp(y) \geq 1+y\) for some suitably chosen \(y\) and the fact that both \(f(x)\) and \(g(x)\) are probability densities and hence they integrate to 1 ]

Now back to the main problem. Since we are free to choose any probability density \(g(x)\) in the inequality (1), let us take \(g(x)\) to be gaussian with mean zero and variance \(k\), i.e., \[g(x)=\frac{1}{\sqrt{2\pi k}} \exp(-x^2/2k)\] The above inequality (1) then simplifies to \[\int_{-\infty}^{\infty}\log(f(x)) f(x) dx \geq \int_{-\infty}^{\infty}f(x)\big(\log\frac{1}{\sqrt{2\pi k}} -\frac{x^2}{2k} \big) dx = \log\frac{1}{\sqrt{2\pi k}}-\frac{1}{2}\hspace{10pt} ...(2)\]

Where we have used the given constraints on \(f(x)\) on the last equation.

Now that's cool because the right hand side of Eqn. (2) is independent of the particular function \(f(x)\) satisfying the given constraints. Which means that, for all functions \(f(x)\) which satisfy the given two conditions, the quantity \[\int_{-\infty}^{\infty}\log(f(x)) f(x) dx \] is at least \(\log\frac{1}{\sqrt{2\pi k}}-\frac{1}{2} = -\frac{1}{2}\log (2 \pi e k)\). So we have a lower bound. However, is the lower bound achievable ? Certainly yes! Because the lower bound is achievable iff we have equality in Eqn (1), i.e. when \(f(x)\) is indeed equal to the zero mean gaussian with variance \(k\). Abhishek Sinha · 2 years, 4 months ago

Log in to reply


Actually I am just a novice to Statistical Mechanics . I only started a bit of light reading on it from this month (specifically after the English exam) .

Currently I am only familiar with the ensemble approaches which use a bit of Combinatorics . But since you are using an integral and that too you intend to find out a function which maximizes the integral , I believe it will take a bit of time and a bit of higher knowledge at that .

If you want an ensemble approach , I can help you out with it or you might also want to look it upon the Internet .

But still , this problem is quite fascinating and I will work on it . First of all I will have to do a bit of reading(of a higher standard) on it though .

Best of luck to you . And do let us know if you make a breakthrough(which I know you will! NSEP Scholar :D) Azhaghu Roopesh M · 2 years, 4 months ago

Log in to reply

@Mvs Saketh I have a strange feeling that it is of the form \(Ae^{-Bx^{2k}}\) or something, possibly \(Ae^{-Bx^{2}}\)... Maybe we can modify value of \(A,B\) so that it satisfies given conditions?? Raghav Vaidyanathan · 2 years, 4 months ago

Log in to reply

@Raghav Vaidyanathan i know the answer bro, it can be found online as well :)

and yes that is the answer, after that, i can impose the constraints and find A and B, infact i have already done that , i want a rigorious clear proof to reach there, please show your method Mvs Saketh · 2 years, 4 months ago

Log in to reply

@Mvs Saketh It was just a guess. The function should be even. It should tend to zero at both infinities. And it should be more than \(1\)for some \(x\). When it is more than \(1\), \(log(f(x))\) will also be positive, hence our integral will increase. When it is less than \(1\), \(log(f(x))\) will be negative and our integral will decrease. We have to somehow make \(f(x)\) increase rapidly and then decrease rapidly to \(0\), hence I suggested exponential. Is it possible to prove this mathematically? I mean, with \(12^{th}\) standard calculus knowledge? Raghav Vaidyanathan · 2 years, 4 months ago

Log in to reply

@Raghav Vaidyanathan good guess, infact, that is one way of proving it, infact it can be shown, that no other function can satisfy it :) and yes, i shall try, thats what i wanted to know actually Mvs Saketh · 2 years, 4 months ago

Log in to reply

@Raghav Vaidyanathan The fact that your guess hit spot on is another indication of your genius !!

Yup! That's a Gaussian function that can be used to determine the probability distribution function but we have to consider a few constraints to it . Mostly because Maxwell sir used a lot of assumptions in his theory . Azhaghu Roopesh M · 2 years, 4 months ago

Log in to reply

Can you please tell me where you saw this, I am just curious to know. Ronak Agarwal · 2 years, 4 months ago

Log in to reply

@Ronak Agarwal alright, i am trying to prove maxwell's speed distribution, for gas

so here is my approach, (do help, if you can spot something in it)

Let me first consider any particular velocity component,

let it be be f(\(v_x\))d(\(v_x\) , let this represent the probability that a particle has velocity lying between \(v_x\) and \(v_x + dv_x\)

(now i shall replace \(v_x\) with x for convinience)

now , the entropy for a system is given as the negative average of the logarithm of probability density and hence i got,

\(S=-\int _{ -\infty }^{ +\infty }{ log(f(x))f(x)dx } \)

additionally i know , that the expected energy of a molecule due to motion along x-axis is \(\frac { kT }{ 2 } \)

and it is equal to \(\\ E=\int _{ -\infty }^{ +\infty }{ \frac { m{ x }^{ 2 } }{ 2 } f(x)dx } \)= \(\frac { kT }{ 2 } \) (the expected energy)

finally, being probability , the integral through out should be 1,

Now, assuming that entropy is always maximised, i have the question,

So i know for sure that if i am able to maximise the integral by appropriately choosing f(x), i will have the expected speed distribution

(i have alternative proofs, but they all start out discrete and then go to continuum limit, and others just mathematically predict the function, but a more rigorious proof would be satisfactory, so please help if you can) Mvs Saketh · 2 years, 4 months ago

Log in to reply

@Mvs Saketh I believe, I don't have the right tactics to deal with such kind of problems. You may ask such questions on physics stack exchange. Ronak Agarwal · 2 years, 4 months ago

Log in to reply

@Mvs Saketh this is just gone over to my head ! But bro I'am just curious to know , are you not preparing for JEE_MAINS ? or you had prepared it very well already ? Karan Shekhawat · 2 years, 4 months ago

Log in to reply

@Karan Shekhawat No no, infact, i am only preparing for JEE right now, all the time, it would be idiotic to sway elsewhere currently , atleast for me....

i just google these interesting stuff in my free time and try to work on them Mvs Saketh · 2 years, 4 months ago

Log in to reply

@Mvs Saketh Instead of using Entropy , how about using a Gaussian Function just like Raghav suggested ?

For example , \(\huge \displaystyle P(v_{x}) = Ae^{-Bv_{x}^{2}}\)

And as per your point , a probability function must be normalised .

\(\therefore 1 = \int_{-\infty}^{\infty} \displaystyle P(v_{x}) dx \)

Then we'll get \(A=\sqrt{\dfrac{B}{\pi}}\) and then I don't know how to follow it up . Maybe take the average value of the mean squared value of the velocity function but wouldn't it lead to us needing the degree of freedom of the gas .

This is just a possibility though .

Is it possible for you to just list out the other methods that you've tried ? I just want to look at it as a source of inspiration .

And I'll also ask some of my other friends about it and let you know :) Azhaghu Roopesh M · 2 years, 4 months ago

Log in to reply

@Azhaghu Roopesh M No, the maximisation of entropy should lead us to the gaussian distribution, otherwise we wont be very satisfied,

Infact, even Maxwell did not derive it using entropy, he very brilliantly proved that only the exponential function can be the function describing the speed distribution, i know that proof, but still

and ,i want to use minimal combinatorics, a combinatorial proof involves assuming discrete levels , and after getting ,answer, making it continuous, i want to avoid that Mvs Saketh · 2 years, 4 months ago

Log in to reply

@Mvs Saketh This is a really interesting problem. And yes, I'm pretty sure it's very difficult too(i.e. don't think I can do it). But please do post the answer here when you end up solving it! Raghav Vaidyanathan · 2 years, 4 months ago

Log in to reply

@Ronak Agarwal @Shashwat Shukla @Deepanshu Gupta @Raghav Vaidyanathan @Azhaghu Roopesh M and any one else who can help, Mvs Saketh · 2 years, 4 months ago

Log in to reply

@Akshay Bodhare Kunal Joshi · 2 years, 4 months ago

Log in to reply


Problem Loading...

Note Loading...

Set Loading...