Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

ACIT4630 - ML/DL > DL-05b - Convolutional neural network - Architectures > Flashcards

DL-05b - Convolutional neural network - Architectures Flashcards

1
Q

DL-05b - Convolutional neural network - Architectures

Who is behind LeNet?

A

Yann LeCun

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

DL-05b - Convolutional neural network - Architectures

When did Yann LeCun publish LeNet?

A

1998

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

DL-05b - Convolutional neural network - Architectures

What was LeNet used for?

A

Reading zip codes/digits.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

DL-05b - Convolutional neural network - Architectures

Who created AlexNet?

A

Alex Krishevsky

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

DL-05b - Convolutional neural network - Architectures

When did Alex Krishevsky publish AlexNet?

A

2012

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

DL-05b - Convolutional neural network - Architectures

What network was published in 1998?

A

LeNet

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

DL-05b - Convolutional neural network - Architectures

What network was published in 2012?

A

AlexNet

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

DL-05b - Convolutional neural network - Architectures

What challenge did AlexNet attempt to solve?

A

ImageNet,

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

DL-05b - Convolutional neural network - Architectures

How many different labels does ImageNet have?

A

1,000

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

DL-05b - Convolutional neural network - Architectures

What’s the architecture of AlexNet?

A

5 conv layers, 3 FC layers, overlapping max pooling

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

DL-05b - Convolutional neural network - Architectures

How many parameters did AlexNet have?

A

~62 million

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

DL-05b - Convolutional neural network - Architectures

When was VGGNet published?

A

2014

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

DL-05b - Convolutional neural network - Architectures

Who published VGGNet?

A

Simonyan and Zisserman

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

DL-05b - Convolutional neural network - Architectures

How many layers did VGGNet have?

A

16, hence it’s often called VGG16.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

DL-05b - Convolutional neural network - Architectures

What was the key innovation of VGGNet?

A

Depth of network is key to performance.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

DL-05b - Convolutional neural network - Architectures

What were some drawbacks of VGGNet relative to its contemporaries? (2)

A
  • more expensive to evaluate
  • requires large memory and computation power.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

DL-05b - Convolutional neural network - Architectures

Whwne was GoogLeNet published?

A

2014

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

DL-05b - Convolutional neural network - Architectures

Who published GoogLeNet?

A

Szegedy et al.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

DL-05b - Convolutional neural network - Architectures

How deep was GoogLeNet?

A

22 layers

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

DL-05b - Convolutional neural network - Architectures

What key innovations did GoogLeNet have? (3)

A
  • Reduced parameters with better performance
  • Inception modules
  • Average pooling instead of FC layers
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

DL-05b - Convolutional neural network - Architectures

What is the inception module?

A

Block that splits into multiple CNN blocks with different kernel sizes.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

DL-05b - Convolutional neural network - Architectures

In inception modules, what do concatenation layers do?

A

Combines all the outputs and feature maps from the conv filters into one single output of the Inception module.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

DL-05b - Convolutional neural network - Architectures

In inception modules, what solves the problem of high computation cost?

A

1x1 conv blocks to reduce number of input channels.

(192 -> 1x1-> 16) -> (16 -> 5x5 -> 32)

(See image)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

DL-05b - Convolutional neural network - Architectures

Describe the full inception module.

A

(See image)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

DL-05b - Convolutional neural network - Architectures

What is depicted? (See image)

A

A full inception module.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

DL-05b - Convolutional neural network - Architectures

What is depicted? (See image)

A

A naive inception module.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
27
Q

DL-05b - Convolutional neural network - Architectures

What are the benefits of using inception modules? (3)

A
  • Extracts features at varying scales using varying filter sizes
  • minimizes computation load
  • utilizes 1x1 conv filters for better cross-channel pattern learning
28
Q

DL-05b - Convolutional neural network - Architectures

When was Inception 4 released?

A

2016

29
Q

DL-05b - Convolutional neural network - Architectures

What tricks are introduced to make Inception better later on? (2)

A

Improvements to speed through:
- (5x5) factored into 2 x (3x3) convs
- (nxn) factored into (1xn) and (nx1).

30
Q

DL-05b - Convolutional neural network - Architectures

When was ResNet published?

A

2015

31
Q

DL-05b - Convolutional neural network - Architectures

Who was behind the ResNet paper?

A

He et al.

32
Q

DL-05b - Convolutional neural network - Architectures

Which company published ResNet?

A

Microsoft.

33
Q

DL-05b - Convolutional neural network - Architectures

How many layers did the original ResNet model have?

A

152

34
Q

DL-05b - Convolutional neural network - Architectures

What idea did ResNet introduce?

A

Residual learning

35
Q

DL-05b - Convolutional neural network - Architectures

What block is this? (See image)

A

A residual block

36
Q

DL-05b - Convolutional neural network - Architectures

What intuition can we use for ResNets?

A

The residual blocks makes the layers comparable to ensembles of different models.

37
Q

DL-05b - Convolutional neural network - Architectures

What are the “two modes” for how layers are trained in ResNets?

A

Layers are either trained or training is skipped for layers using skip connections.

38
Q

DL-05b - Convolutional neural network - Architectures

When were Inception-ResNets published?

A

2017

39
Q

DL-05b - Convolutional neural network - Architectures

Who were behind Inception-Resnets?

A

Szegedy et al.

40
Q

DL-05b - Convolutional neural network - Architectures

What is SENet short for?

A

Squeeze and Excitation Networks

41
Q

DL-05b - Convolutional neural network - Architectures

Who were behind SENet?

A

Hu et al.

42
Q

DL-05b - Convolutional neural network - Architectures

When was SENet published?

A

2017

43
Q

DL-05b - Convolutional neural network - Architectures

What does a squeeze-and-excitation block attempt to do?

A

Enables the network to perform dynamic channel-wise feature recalibration.

44
Q

DL-05b - Convolutional neural network - Architectures

What is the name of the block that attempts to perform dynamic channel-wise feature recalibration?

A

Squeeze-and-excitation block

45
Q

DL-05b - Convolutional neural network - Architectures

In a squeeze-and-excitation block, what is the purpose of squeezing?

A

squeeze global spatial information into a channel descriptor.

46
Q

DL-05b - Convolutional neural network - Architectures

In a squeeze-and-excitation block, what is the purpose of excitation?

A

Excitation operation is used to fully capture channel-wise dependencies.

47
Q

DL-05b - Convolutional neural network - Architectures

Squeeze-and-excitation block looks like?

A

(See image)

48
Q

DL-05b - Convolutional neural network - Architectures

What is the technique used in EfficientNet called?

A

Compound scaling

49
Q

DL-05b - Convolutional neural network - Architectures

Which network introduced Compound scaling?

A

EfficientNet

50
Q

DL-05b - Convolutional neural network - Architectures

What is Compound scaling?

A

An equation for setting the width, depth and image resolution of a network.

51
Q

DL-05b - Convolutional neural network - Architectures

What is the equation for compound scaling?

A

(See image)

52
Q

DL-05b - Convolutional neural network - Architectures

What is MBConv short for?

A

Mobile inverted bottleneck

53
Q

DL-05b - Convolutional neural network - Architectures

What is the shortened name of Mobile inverted bottleneck?

A

MBConv

54
Q

DL-05b - Convolutional neural network - Architectures

What is the main idea behind MBConv?

A

They use an inverted structure for efficiency reasons.

55
Q

DL-05b - Convolutional neural network - Architectures

What does MBConv blocks do? (3 steps)

A
  • Increases number of channels with 1x1 conv
  • Depth-wise convolutions
  • Downsample channels to initial value using 1x1 conv
56
Q

DL-05b - Convolutional neural network - Architectures

Who is behind EfficientNet V2?

A

Tan and Le.

57
Q

DL-05b - Convolutional neural network - Architectures

When was EfficientNet V2 published?

A

2021

58
Q

DL-05b - Convolutional neural network - Architectures

What does EfficientNet V2 use? (3) (BLR)

A
  • Combines MBConv and Fused-MBConv blocks
  • Uses progressive learning (changes image size dynamically)
  • Uses adaptive regularization
59
Q

DL-05b - Convolutional neural network - Architectures

How does ResNet tackle the error propagation challenge?

A

Through skip connections.

60
Q

DL-05b - Convolutional neural network - Architectures

What makes error propagation efficient in a ResNet architecture?

A

Residual connections allow error to bypass layers, ensuring efficient error propagation and training.

61
Q

DL-05b - Convolutional neural network - Architectures

What architecture is this? (See image)

A

MBConv

62
Q

DL-05b - Convolutional neural network - Architectures

Label the masked parts.

A

(See image)

63
Q

DL-05b - Convolutional neural network - Architectures

Describe the full architecture of MBConv.

A

(See image)

64
Q

DL-05b - Convolutional neural network - Architectures

What architecture is this? (See image)

A

Fused MBConv

65
Q

DL-05b - Convolutional neural network - Architectures

Label the masked parts.

A

(See image)

66
Q

DL-05b - Convolutional neural network - Architectures

Describe what parts a fused-MBConv consists of.

A

(See image)

ACIT4630 - ML/DL (18 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions