NELL for a puny human being

 I miss those days when I did not have to repeat things to memorize them. Recently, I have been trying to seriously learn french and my best language learning years and behind me by some 25 years. Still, one gotta do what one gotta do, and in a stable peaceful manner.

Learning is fun, and I am trying to find the best strategy for it these days. Productivity is a metric for machines and not humans and while it is important to build the world and oneself, it should never ever be the main goal. Productivity is what I associate with the workplace at most- these guys pay me and I owe it to them to deliver maximum output for the time they have engaged me. The definition of output has to align a lot with their definition, but it is the only place in the life of a human being where it matters.

Also, I should take a one week vacation from office and spend ten days entirely cut off from the world, with absolutely zero communication. The joy it is going to bring mw can only be paralleled by a few things in life.

Back to being a cog-I am a cog for financial reasons, but I have a life of my own too. So-NELL. Mitchell's research sounds very fancy and important, so I have never dared look closely into it, but I can toy with the idea.

Reading books about fun stuff is fun but acquiring knowledge requires some structure to retain it. So I think it is important to build one's own database and to make original contributions after acquiring a certain level of skill. There has to be some effort made into showing it to the world since the populace at large deserves to benefit from an exchange of ideas, but not too much because it is an extremely stupid goal to pursue. I am an extreme fan of people like Azad who are scholars through and through and it is their only passion in life. More than being a fan, I relate to it. The wanting part though, and not the achieving one.

So there are multiple questions around this problem: how does one select an area of study? Should one work on one area, or multiple in parallel? After reading something, how do you make sure that you retain something from it? Memory is notoriously bad, and mine is almost non-functional these days. There are a lot of solutions, and the search for the perfect one takes forever, so I'll have to settle for some sub-optimality. I guess I am going to start noting things down on paper. Summaries, so to speak.

As for single vs multiple, I really can't say. Right now I have my eye on a few things, all of them theoretical of course but that can't be helped at least for now. For now the solution seems to be to choose a few and stick to them. Focus is key, so I think I can do one thing a day, spend a few hours on it, but I'll have to make sure the work gets done with extreme focus. There is no point in doing things half-heartedly-watching youtube would be a better alternative.

I would like to read stuff up in the following areas:

1. ML

2. Philosophy

3. French

4. Poetry

5. Politics but maybe

6. Filmain, cause I love them

Okay I think we can skip politics(or serious study thereof) for now, but I'll be working on the rest of them. If I work on my so called research a lot of time gets diverted to ML but that is life for now, and I need this work for my backup plans too.

If I ever get the chance in the future, I might pick up something different like fencing or kathak but these are my only options for now.

It seems to me that writing stuff is a fundamental part of the learning process and just like it is needed to learn decision theory, it is needed for philosophy and poetry too. I might be reporting here to mention what I have learned in a day. I am a bookish bore and I am not going to try to escape it.

A peaceful saturday and a possible flu

 I discovered my red nail polish and I am very ambiguous about the shade. The day was nice for a change-probably I have calibrated myself well to continuous zalalat-but I have a slightly sore throat and I just hope it does not develop into full blown flu.

I am waiting for rejections but they are usually slow so I don't even have anything to look forward to.

I have serious backup plans to make, but I don't even have an inkling of where to start.

Ah well I will figure it out. In the meantime, I have some decision theory to study.

All along, I have been right and people have been-silly. Ah well people are people, can't expect them to understand anything beyond their own petty interests.

I have petty interests too, but I'm unable to take care of them, so poor them are just left in the void fending for themselves. 

Depending upon me is always a bad idea.

For fourteen programs, if I guess the order in which I am going to get rejected from them, what would be the probability of me being right?

What is the probability of me being right, ever? Exactly zero.

Exactly.

Still one stupid letter left.

I am also reading an interesting story. 

Man, age makes one mellow.

There should be a glue for nails. Natural.

Er Turan makes for nice background music for study.

I am rather tired of one thing after another holding me back. I am done with it. However, as with most of my decisions, this one is not very likely to get translated into anything.

I don't know what is the thing with principles. People usually don't have any. Most of the ones they have are adopted on an ad hoc basis, to serve self interest. I have no issues with anyone acting in their self-interest, but I rather wish people would be upfront about it. As for me, I need them more for theoretical reasons than anything. Otherwise it becomes difficult to act independently all the time.

If only I could seriously study philosophy.

Well I can, if I want. Maybe I just don't want it enough. I am done with overthinking, and procrastination is next. Laziness and ignorance too. Being ignorant is inevitable, for there are infinite many things to know, but I am going to die fighting it.

Life requires a very specific level of intensity, one that I often have trouble regulating. Ah well, one of these days.

Bracing for an entire year of zalalat

 I was in an okayish mood when I got up at 12, but then I saw a multi-point reprimand by my boss and of course that was the start of a downward slide. Still, I am trying to rationalize it, since he is right and I just suck at my job. I am sure there is some self-sabotage at play here too and I do not relish the idea of getting fired; in fact, it would be a tangible loss and a problem for me, were it to happen, but I have been robbed of the ability to give a damn.

Still, this is a realistic forecast for the year, and I should not be surprised if the following happen:

1. I get rejected from all 14 places. In fact, it is quite likely. 

Okay I just got a new year package from my office. I love these guys, and feel sorry for them.

Back to the incoming wave of rejections. I have one letter pending now, but the process was and continues to be draining. You hear all these things about how hard it is to get into NLP and then I am a weak candidate by so many definitions.

I check gradcafe from time to time, but I have a story and some decision theory to read so it isn't frantic scrolling. Anyways I could never compete with the world.

I hope I can somehow make it to darya kinaray. If it is a low stress place, I might even be at peace there. If there is an actual darya there, I'll visit it in the evenings and read poetry.

I wonder if I can make it to a single competitive program, but maybe I am entirely unsuited to these programs.

2. I get fired from my job. Pretty self explanatory at this point.

3. My mental balance deteriorates even further. However, I will be working really hard to prevent it. Right now, it is the only thing that matters and I can go to extreme lengths to protect it. For maybe the first time in my life, I actually mark topics as not-to-be-considered. Like I simply don't care. Feelings are only a matter of training your own grey matter and I can and will do it.

4. Bad stuff happens. A category as broad as broad can be.

Agonizing over consequences just gives one a pathetic digestive system, something that I am tired of. I need to start marking things as binary, in my control and not in my control. If they are in the former, I will think and adopt the best strategy. The latter will just get banished from my mind; them pretty neurons deserve stuff that wants to be there. 

My writing is full of non-sequitrs, just like me, and it makes writing academic papers hard. Writing papers is a hard skill and it will take me quite some practise to get something done.

I hope I stick to decision theory and see it through.

Emotion regulation is the only thing I really need to work on. The rest of life just falls in place.

If I happen to die in this year in an abrupt, non-dramatic manner, I'll consider myself the luckiest of creatures. The only good thing that can happen to one in any given year.

People suffer and toil a lot in the world, and I get sick of my own whining. Still, this is something I have been writing here for the past ten years, so the realization has never been the problem. It is the follow up work that I have problem with.

I wonder what it is with the human need to form and belong to communities. Maybe everyone is just looking for information to maximize their own gain. Maybe this is the sole reason and I have mistakeny been looking at a longing for belonging as the cause. People socialize because it is useful.

Well, I can socialize. During 2020, I was rated as 2/10 on work and negative infinity on charm among a group of women that extends from sahiwal to seattle. This year I'll be running a psychological experiment in my free time, as much of it as can be spared, that seeks to rank my attractiveness. Experiment design is a science but what people don't realize is that I formally studied it in grad school, and it is high time I had a reliable estimator for this problem.

Life is a cold waiting room

 Life is a cold waiting room these days, and the liquid nitrogen I had had pored over me recently only makes it colder, although it should be the opposite.

Living day to day is an interesting experience. Sometimes you laugh, sometimes you cry. 

The mornings are mostly sunny, so I have been baked to quite some extent.

Last night I started reading something that has memory and empire in its name. It is interesting, and might be better than rhythm of war. My taste has changed, for sure. I have been saying that I want to read fiction, so fantasy might work.

Gradcafe has its allure, no doubt. Two of my letters are still pending and it is a cause of distress.

My brain refuses to let go of things, and I am often in a sickening state of mind, but where can I lodge a complaint?

I am waiting for life to happen to me.

I need to start practising kindness to myself, and to others. Only recently have I realized the value of it.

Should check gradcafe again.

Another cold and windy morning, but unhappy on top

 So the morning was somewhat more sunny but I just couldn't bring myself up from some depths of melancholy so there's that.

Just checked gradcafe and my email, nothing interesting anywhere.

I should be working but as usual I don't feel like it. I did study some theory of empirical risk minimization.

I have been feeling like crying since the morning, but it isn't going to help.

Someone advised me to just stop thinking about things, and I'm trying that.

I wish I could get a coffee from crazy mocha. Not gonna happen right now.

There should be a simple matrix for describing how one is feeling at any given moment.

I guess I'm thinking about robustness of deep neural networks all the time these days. Yesterday I read the original paper by Szegedy, and it was a nice piece of work. They hypothesize that these are due to statistical nuances of the training dataset. I find the whole thing fascinating from the point of view of being clear about what a trained neural network can and can't do. Also I think they represent generalization errors, as in the network learns to associate something random with a particular label and carries that error forward in its lifetime of classifications. What does the decision surface of a neural network look like is another question we should be asking ourselves. For images, since random and small perturbations cause misclassifications, it seems that the decision surface is extremely jagged.

Another question we should be asking ourselves is whether the increase in performance due to, say, a larger network, comes at the cost of generalization. Sure, you can use validation error for early stopping, but some generalization errors are likely to reward you on the validation set too. I am not very familiar with vision problems in general, but I do think that large networks are extremely extremely overparameterized. One thing I noted is that Sazagady mentioned that adversarial examples transfer to models that have been trained on a different subset of training data, but the success there is very small and further references ignore this fact.

Overall, I am in favour of developing global techniques for interpreting a model, but I don't know what the current state is in computer vision. After training, the model should provide a summary of what it has learned to associate with each class, and what is anti-correlated. NLP problems are surprisingly simpler, despite what we claim in our work, but my lack of access to a GPU doesn't let me work on vision. 

Another interesting line of thought stems from the fact that data has some real world footprint, and we should be aware of it in our practices. For images, I am very keen on observing the effect of perturbing edges on network output, but there doesn't seem to be any correlation. Ugh!

For a fixed input CNN, theoretically we can test all different permutations of input on the model and observe its output. For discrete input/discrete output systems, this input/output map would be a valid summary. Pixel values aren't really continuous and there is a theoretically finite number of inputs you can provide. Sampling from this distribution is another problem however.

A cold and windy morning

The title says it all, at least 'all' that is on the surface. 

Surprisingly, I feel somewhat better this morning, maybe because I slept relatively well. The cold air is nice and I came across a strange but nice song that depicts a battle in a desert.

Recently I have started realizing the true extent of my ignorance. However, there isn't much to do, since my cognitive functioning seems to have reached close to zero. Since it was something I depended on a lot, navigating life without it is tricky. I do not know if it is temporary or permanent, but I have to make peace with life as it is.

We all have to.

My mental balance questionnaire is still pending. Ah well, one of these days.

For the past few years, I have been living under too much stress. I have edged very close to the tipping point at times, but something or the other pulls me back. Still, it is a dangerous place to live, with very real physical consequences. Reading something this morning made me realize that losing grip on reality is a hallmark of mental illnesses, and I constantly complain about mine. It should have alerted me a long time ago that something was terribly amiss.

There are things one has to accept, and things one has to let go of. My time circuit is very strange and I have a very very weird sense of the flow of time. Ultimately you have to sculpt your own balance-I never bother myself with things like happiness and joy, I was not made for them and they were not made for me-and it is an ongoing quest. I can complain about it taking up too much of my energy, but who is going to listen to it. All I need are practical techniques to manage my situation, and the energy to carry them out.

Energy is a highly variable variable. Some days I feel like doing a mild amount of work, and do it. Some days all I can think of is my bedtime, when I will be able to curl up in my bed and cut myself off from the outside world, ever since I get up. It becomes very hard to do anything at all and I constantly worry about my job, as I should, for it is in reality the only tangible and solid relation I have in my life.

Music like this makes me want to get up and conquer the world, but then I think of the effort involved and just turn over in my bed. Motivation is a complex phenomenon, and now I think that emotions are a huge component of everything. A necessary ingredient. For doing anything, you need to have the right mix of curiousity, happiness, stability, fear, chance of reward and the like, and if any component is lacking you simply can't start the work. Fortunately or unfortunately, I can never work under a negative frame of mind. Threats and fears and stress make me lose all motivation and all productivity. Heck, they make me lose the impulse towards life itself.

At this point and age, I have no ambition and no dreams, besides being able to sleep eight hours straight. I check gradcafe everyday, although I have mixed emotions about it.

If there is one thing I have learned recently, it is this: the world and people will inevitably be cruel to you, so make sure you aren't cruel and cold to yourself. Indulging in self pity is not something I condone, but at least you should have a healthy respect for yourself, and recognize when you have been wronged, and sympathize with yourself, and cut yourself some slack from time to time.

And stop thinking of life as a goddamn race. We are all ultimately racing towards our graves and whatever we pick up on the way, or don't, isn't really relevant.

The trouble is, my brain has this tendency to stick to something that it becomes impossible to let go of it. Of course I try, but it is another constant battle. The best thing to do is to provide it interesting stuff to work on.

Living in imagination for three days

 So recently I have figured out that all my problems stem from me being me and if I could stop being myself, they would all pause for the said duration.

Yesterday I was sure that I was going to get rebuked on something that recently happened but no one said a word and it made me realize that my grip on reality is really really tenuous. So I decided to take it further.

Side but important note, I am addicted to checking gradcafe everyday. 5 of my letters are pending, and one at a nice place.

I think I am going to put Karpathy's advice into practise.

Back to main track.

I dreamed up a character, a princess called zartashiya, who got sick of the squabbles of her ministers and the ever-going crises in public life and decided to leave her life of power and strife for one of peace and knowledge. She handed over the task of ruling to her younger brother, secretly left the palace and made her way to a farm in a valley. She dwells there in peace now, pursues gardening in the morning and her studies in the evening. She is a kindly soul, though with her own quirks, and she does not talk a lot to people. However, she is kind to all who have dealings with her, and rather sparse in discourse. She laughs often and smiles seldom, and derives her joy from the earth and from the wisdom of words, though the former is eternal and ageless and does not speak and the former come from people who are dead, have been dead for centuries and eons, but their ideas and thoughts, conveyed through the trusted medium of paper, make their way to her heart in a way that words from boyses never do. Discriminating and labelling people is often done for the ease and convenience of a naive labeller, and the fact of them and her being separated by time is of no great consequence to her. She is content with herself, time, and the world, and not a syllable of complaint ever rises to her lips. If you ever have the fortune to be granted audience before her, you will find her wise and understanding, and she will listen to you more with her gleaming eyes then anything else, and her words will comfort and soothe you if nothing else.

Being a scholar of human nature, we have a few questions to ask. What exactly is it that makes her happy, that makes her content? Is it because she has been seen and dealt with ambition, and power? Is it because she has left everything of her volition, and chosen her path for herself? Does this show an absolutely essential but corrupt and corrupting side of ambition, or is it just because her nature is far more suited to the farm than to the palace? Do we believe in a human nature that is organically more suited to some things than other? 

I have had the chance to listen to a few things about emotions, how Hume believed that an emotion is an idea that is attached to an effect, and Spinoza believed the opposite, that idea is supreme. I think that an emotion is a learned response which you train yourself to produce, as evidenced by the fact that emotions are never eternal and always fade over time. There are multiple layers of thoughts behind every emotion, because every emotion is tied to multiple things, and thus maybe resides in multiple areas of the brain, but they can always be tackled one by one.

I have been thinking about formulating a questionnaire for determining how mentally stable one is at any point in time. It will be a rough metric, but better than none. I wish they made equipment for measuring mental balance.

جاتے جاتے اس کا وہ مڑ کر دوبارہ دیکھنا