Research Methods and Tools Flashcards
How would you define research?
- Studious inquiry or examination; especially: investigation or experimentation aimed at the discovery and interpretation of facts, revision of accepted theories or laws in the light of new facts, or practical application of such new or revised theories or laws.
- The collecting of information about a particular subject.
Constituents of research
- Defining, redefining, and/or formalizing problems.
- Formulating hypotheses.
- Suggesting solutions or solution approaches.
- Collecting and analyzing data.
- Experimenting.
- Deriving new knowledge and/or formulating new theories.
What is NOT research?
- playing with technology;
- developing code;
- deploying standard or commercial technology;
- doing what others have already done.
Why are you doing research?
- Meaningful and long-lasting contributions towards the advancement of mankind and society.
- Attain a higher level of understanding of fundamental concepts.
- Intellectual satisfaction provided by doing something innovative and creative.
- Enjoy the challenges of solving unsolved problems.
- Degrees, financial benefit, and respect all come along the way.
Different types of research:
- Descriptive: surveys, comparative and correlational methods.
- Analytical: analyze and critically evaluate information.
- Applied: address practical problems and solutions that can be implemented for near-term benefits.
- Fundamental: generalization and formulation of theories.
- Quantitative: numerical results are used to validate claims.
- Qualitative: comparative development of usage patterns and experiences.
- Conceptual: relies on abstract ideas or theories.
- Empirical: relies on experience and observations.
What is Computer Science?
- The discipline of Computing is the systematic study of algorithmic processes that describe and transform information: their theory, analysis, design, efficiency,
implementation, and application - Computer Science is the study of phenomena related to computers, information structures, and management of complexity.
what are Computer Science branches?
Theoretical Computer Science:
- Builds theories as logical systems with the aim of deriving/proving theorems.
- Design and analysis of algorithms.
- Understanding the limits of computation.
- Distillation of knowledge acquired through conceptualization, modeling, and analysis.
Empirical (Experimental) Computer Science:
- The field of inquiry is the nature of information processes.
- Used for theory testing and explanation, unearth new phenomena that need explanation and help derive theories from observation.
How to Select PhD topic?
- Understand your expertise and limitations.
- Focus first on breadth of knowledge; depth comes later.
- Be specific about the topic, but be flexible about the scope.
- The topic should be rich enough but also accessible.
- Be prepared to revisit the topic you selected, if needed.
How to build your foundations?
- You should have or build a very strong background and foundations on the broad area of your topic.
- You should have some ideas about the state of the art.
- You should like and enjoy the chosen topic.
- Choosing a new topic vs. an old topic.
- You need to envision the future prospects of your intended topic (e.g., working on a standard that later is dropped)
How can you Expand your knowledge?
- You should expand your breadth of knowledge on the selected topic.
- Read the fundamentals on the topic to build up the foundations for your research.
- These efforts will pay off in the long run, even if it does not seem so initially.
- Feel open about broadening the scope of the topic as you build up on it.
Why do we need to read papers?
- to cultivate knowledge of the area.
- to learn about recent advances.
- to avoid reinventing the wheel.
- to place our work in the proper context (related work)
A careful selection of papers saves a lot of time because there are several must-read papers in every area, including “historical” papers and a lot of “garbage” around.
Three-pass approach for reading papers
- Quick scan (5-10 mins):
- To decide whether the paper is worth reading at all.
- Reduces significantly the number of papers to process further. - Reading with greater care (1 hour):
- Helps in grasping the content.
- Helps to better understand the contributions of the paper. - Detailed reading (4-5 hours, but may take much longer):
- To fully understand the paper.
- Helps in identifying open issues and ideas for future work.
Explain the three pass to read papers
Pass 1:
- Carefully read the title, abstract, and introduction.
- Read section and subsection headings, but ignore everything else.
- Read the conclusions.
- Glance over the references.
- After reading you should enable to answer these questions:
Category: What type of paper is it (experimental, system descr., …)?
Context: To which papers is it related? What bases were used?
Pass 2:
- Read with greater care, but ignore details (e.g., proofs).
- Identify areas of your interest.
- Identify results relevant to the scope of your topic.
- Scribble in the margin important points, thoughts, and questions.
- Mark relevant references for further reading.
- After reading you should be able to:
Grasp the content of the paper.
Summarize main contributions, with supporting evidence.
Pass 3:
- This is required to fully understand the paper, especially if you have to review it.
- This requires great attention to detail.
- Think about how you would present a particular idea.
- You can learn new techniques and good presentation style.
- Note down open problems and ideas for future work.
Where to publish your paper or research?
Publishers:
- reputation.
- professional societies.
- stay away from private commercial publishers (predatory publishing!).
Quality and metrics of journals and conferences:
- impact factor.
- citations, bibliometrics.
- accessibility to researchers.
- Conference ranking: http://valutazione.unibas.it/gii-grin-scie-rating/ (by GII+GRIN+SCIE)
- Journal ranking: https://www.scimagojr.com/ (based on Scopus database by Elsevier).
Electronic publishing.
Make your work available online.
Citations and impact factors (IF)
- Citations quantify the impact of a paper.
- The number of citations is a measure of how well known the paper is, and how significant the results are.
- Impact factor (IF) is used mostly for journals and is defined as the average number of citations for its papers.
- Impact factor is sometimes misleading (e.g., predatory publishers force citing papers in their journals).
Different types of papers
Different kinds of papers serve different purposes, and are directed at different audiences:
- Paper abstracts
- Extended abstracts
- Research papers
- Position papers
- Survey papers
- Magazine articles
Workshops, conferences, journals
In computer science, when we publish our original research, we often follow the 3-phase model:
1. one or more workshop papers with initial ideas and preliminary contributions.
2. one or more conference papers, each providing original, substantial results.
3. a journal paper, that consolidates, and expands the original research contributions.
Conference vs. journal publications
Refereed conference publications with a high value are a peculiarity of CS.
Conference rankings are being established, but are still controversial.
The number of conferences has increased dramatically, at the price of overall quality (Disadvantages of publishing in conferences) :
- Too many conferences (and journals) that accept low-quality papers.
- The reviewing load has increased, there is less time for reviewing, and reviews are shallow.
- Predatory conferences that accept everything without proper reviews.
From conference to journal paper
Different models are being adopted:
- Revision of the paper with 25%-30% new material.
- Disadvantages: citation splitting, lag time. - Journal first, with a paper published in a journal, and authors invited to present at the conference.
- Becoming increasingly popular. - Journal-integrated, where papers accepted in the conference review process are published in the journal, and presented at the conference.
- Papers that need additional reviewing are transferred to the journal review process.
The scope of a paper
Key aspect: What to include and what not in a paper?
Typical questions to ask:
- Which results are the most surprising, original, and technically challenging?
- What might other researchers adopt in their work?
- Are the other outcomes independent/interesting enough to be published separately?
- What is the key background work that I need to discuss to explain my novel contribution?
- Which preliminary material should I include to make the results understandable?
- How much detail of the novel contribution should I include in the paper?
Choosing the right venue
The scope is largely determined by readership and venue.
Typical questions to ask:
- How relevant is the topic for the venue?
- How does my work measure against the standard for that venue?
- Are there page limits to consider? (There always are!!!)
- What is the background of the typical reader?
- Are proofs of theorems required/expected/desired?
- Is an experimental evaluation required?
- Is the deadline compatible with the work plan?
- When is the next deadline for an appropriate venue, if the upcoming deadline is missed?
Telling a story
- An effective paper educates its readers.
- It leads them from what they already know to new knowledge you want them to learn.
- Hence, the body of a paper should have a logical flow that has the feel of a narrative:
It is a walk through the ideas and outcomes, it should not explore all wrong attempts and unsuccessful paths. There should be a logical closure, where the hypotheses or claims presented in the paper are shown to be justified.
Developing the story
As a chain, the results and the background on which they build dictate a logical order for the presentation of the material (e.g., problem statement, previous solutions are bad our new solution is better).
- By specificity, suited for results divided into stages.
- By example, proceeding from a specific instance to the general framework.
- By complexity, proceeding from simpler to more complex cases.
Typical constituents of a paper
- Title and information about authors
- Abstract
- Introduction
- Body
- Related work
- Conclusions
- Bibliography
- Appendices
Title and information about authors
- Papers begin with title and information about authors including name, affiliation, and address: Use always the same spelling for your name. Use a durable email address, but prefer your institutional address.
- The title is very important. It is read by thousands of people. A paper with a bad title might not be found and read (Not too short or too generic, not too long or specific, but specific enough to differentiate it from similar work).
- How to list the authors? whoever gave a substantial contribution to the research (contributed to the actual writing of the paper- proved a key result or carried out a key experiment- implemented specific software that is necessary), the Order of names is important.
Abstract
- Is typically a single paragraph of 50-200 words.
- Allows readers to judge the relevance of the paper to them.
- Is a concise summary of the paper’s aims, scope, and conclusions.
- Should be as short as possible while remaining clear and informative.
- The more specific, the more interesting.
- Use past tense, since it refers to work already done.
Introduction
- Can be regarded as an expanded version of the abstract.
- Should describe the paper’s topic, the problem being studied, references to key papers, approach to the solution, scope and limitations of the solution, and outcomes.
- Explanation of the specific problem (difficulty, obstacle, challenge) to be solved.
- Summary of how the solution was evaluated and what the outcomes of the evaluation were.
- There needs to be enough detail to allow readers to decide whether or not to read further.
AVOID: supporting evidence, unnecessary jargon, complex mathematics, in-depth discussion of the literature
Body
- Provides necessary (formal) background and terminology.
- Defines the hypothesis and major concepts.
- Explains the chain of reasoning that leads to the results.
- Provides the details of central proofs.
- Explains the experimental setup and summarizes the outcomes. - States in detail and analyses the results of the research.
Related work
Most results are additions to existing knowledge.
A literature review is used to:
- describe existing knowledge,
- compare the new results to similar previously published results, and explain how the new results extend existing knowledge.
Where to place the literature review?
a) Early in a paper, to describe the context of the work:
might then be part of the introduction.
b) After the main body: allows for a detailed comparison between old work and new results.
c) Along the paper, where it is used:
background material in the introduction analysis of previous results, where the own results are presented.
Conclusions
- Are used to draw together the topics discussed in the paper.
- Should include a concise statement of the paper’s important results and an explanation of their significance.
- Hence, write “Conclusions”, not “Conclusion”. If you have no conclusions to draw, write “Summary”
