Do you have any suggestions on how to solve challenges in a new way? Whenever I believe I have something, my adviser always says something like, “This is too straightforward.” Many approaches combine pre-existing building elements in unexpected ways, but I find it difficult to envision how individuals may create really innovative and functional solutions.
Sometimes when I read a paper, I discover that the writers are merely introducing a neat gimmick, and the notion is really extremely basic or plain. At other times, they observe something I could only imagine thinking about, or I read something that presents a really esoteric theory. Although I lean toward the former group, my lack of originality has prevented me from feeling particularly pleased of anything I’ve written so far. It doesn’t help that I tend to favor “simple yet effective” approaches because of the crazy speed of publication, when the majority of the effort is done after obtaining SOTA to build a tale.
7 Likes
In my situation, it went like this: I had a goal that I want to accomplish. My goal was to implement diffusion models in an environment that had not before been done so. Additionally, I had no clue how to accomplish it, so I prepared a ton of material until one article in particular provided me the solution. And it was successful.
Thus, a little bit of luck and perseverance are needed. Understanding the current status of your field of study is also crucial.
In the end, it was my idea. wasn’t very novel; it had been published a year earlier in a different context.
This is machine learning’s initial rule. Whatever you do, someone has already
4 Likes
Since many of the finest papers are observational, or you could take an existing algorithm and establish its qualities, you can perform research without pursuing SOTA. This is really a rather wide subject.
There is a trade-off between novelty and ease of accuracy improvement if you still want to create new algorithms. There is minimal innovation, yet on the one hand, you want to take the current SOTA approach, add a neat twist, and “easily” earn your 1% increase. However, performance is terrible even if you have a completely different notion. For this reason, the majority of machine learning papers at regular conferences perform the following: 1) They make their contribution seem very original in their paper in their manuscript, they pretend their contribution is very novel by clever marketing and useless math 2) in their code, it’s the same as the previous SOTA baseline with 1 line changed to add their trick.
4 Likes
In my situation, identify a problem, establish a specific goal, optimize toward it, and then, in due time, new approaches will emerge to assist resolve the issue.
3 Likes
The finest ideas, in my opinion, are simple yet efficient techniques. Sometimes people go too far when there are more effective ways to handle an issue.
Putting rants aside, the greatest approach to this is to have a thorough understanding of an issue and then attempt to solve it. Making the leap from picture production to video/language multimodality will never provide really innovative ideas. You will come across them, for instance, when you are making a concerted effort to apply diffusion models to divide two guitar tracks. If you are interested in this, change your PhD to something else, like contemporary hopfield networks, since you won’t find these innovative concepts operating after SOTA. Where there are uncharted and undeveloped fields
2 Likes
Your main problem seems to be that you need a research plan. Choose a problem to strive toward. An idealistic feeling that seems beyond the realm of practical attainment. Concentrate on little steps that contribute to your larger plan for fixing that moonshot challenge.
1 Like
Sounds like your supervisor is part of the problem we have in machine learning, where fancy novel methods are often favoured against simple and straightforward methods, which often perform better despite - or maybe even because of - their simplicity.
IMO there is not a direct recipe to follow, it often starts with noticing any kind of issue in current methods, be it that they are not applicable to certain settings, have some failure modes or even that there is no method for some use cases. Analysing why it doesn’t work yet often helps with coming up with ways to make it work. And this can often be a very simple trick.
Compile a list of items, identify those that haven’t been merged, and attempt to determine how they may be combined.