资源描述
Lecture 8:GenerativeAdversarial NetworkLecture 8:Generative12November 27,2019Artificial IntelligenceGenerative Adversarial Networks Genarative Learn a generative model Adversarial Trained in an adversarial setting Networks Use Deep Neural Networks2November 27,2019Artificia2Artificial Intelligence3Generative ModelsNovember 27,2019Artificial IntelligenceGenerat3Artificial Intelligence4Generative ModelsNovember 27,2019Artificial IntelligenceGenerat4Artificial Intelligence5Why Generative Models?Discriminative models Given a image X,predict a label Y Estimates P(Y|X)Discriminative models limitations:Cant model P(X)Cant generate new images Generative models Can model P(X)Can generate new imagesNovember 27,2019Artificial IntelligenceWhy Gen5Artificial Intelligence6Magic of GANsNovember 27,2019Artificial IntelligenceMagic o6Artificial Intelligence7Magic of GANs Which one is Computer generated?November 27,2019Artificial IntelligenceMagic o7Artificial Intelligence8Magic of GANsNovember 27,2019Artificial IntelligenceMagic o8Artificial Intelligence9GANs ArchitectureNovember 27,2019Artificial IntelligenceGANs A9Artificial Intelligence10November 27,2019Adversarial TrainingAdversarial Samples:We can generate adversarial samples to fool a discriminative modelWe can use those adversarial samples to make models robustWe then require more effort to generate adversarial samplesRepeat this and we get better discriminative modelGANs extend that idea to generative models:Generator:generate fake samples,tries to fool the DiscriminatorDiscriminator:tries to distinguish between real and fake samplesTrain them against each otherRepeat this and we get better Generator and DiscriminatorArtificial Intelligence10Novem10Artificial Intelligence11Training DiscriminatorNovember 27,2019Artificial IntelligenceTraini11Artificial Intelligence12Training GeneratorNovember 27,2019Artificial IntelligenceTraini12Artificial Intelligence13Mathematical formulationNovember 27,2019Artificial IntelligenceMathem13Artificial Intelligence14Mathematical formulationNovember 27,2019Artificial IntelligenceMathem14Artificial Intelligence15Mathematical formulationNovember 27,2019Artificial IntelligenceMathem1516November 27,2019Artificial IntelligenceMathematical formulation16November 27,2019Artificial16Artificial Intelligence17Advantages of GANsNovember 27,2019Artificial IntelligenceAdvant17Artificial Intelligence18Problems with GANsNovember 27,2019Artificial IntelligenceProble18Artificial Intelligence19Problems with GANsNovember 27,2019Artificial IntelligenceProble19Artificial Intelligence20November 27,2019FormulationDeep Learning models(in general)involve a single playerThe player tries to maximize its reward(minimize its loss).Use SGD(with Backpropagation)to find the optimal parameters.SGD has convergence guarantees(under certain conditions).Problem:With non-convexity,we might converge to local optima.Artificial Intelligence20Novem20Artificial Intelligence21November 27,2019FormulationGANs instead involve two(or more)playersDiscriminator is trying to maximize its reward.Generator is trying to minimize Discriminators reward.SGD was not designed to find the Nash equilibrium of a game.Problem:We might not converge to the Nash equilibrium at all.Artificial Intelligence21Novem2122November 27,2019Artificial IntelligenceNon-Convergence22November 27,2019Artificial22Artificial Intelligence23Problems with GANsNovember 27,2019Artificial IntelligenceProble23Artificial Intelligence24Mode-CollapseNovember 27,2019Artificial IntelligenceMode-C24Artificial Intelligence25Some Real ExamplesNovember 27,2019Artificial IntelligenceSome R25Artificial Intelligence26Some Solutions Mini-Batch GANs Supervision with labels Some recent attempts:Unrolled GANs W-GANsNovember 27,2019Artificial IntelligenceSome 26Artificial Intelligence27Basic(Heuristic)Solutions Mini-Batch GANs Supervision with labelsNovember 27,2019Artificial IntelligenceBasic2728November 27,2019Artificial IntelligenceHow to reward sample diversity?At Mode Collapse,Generator produces good samples,but a very few of them.Thus,Discriminator cant tag them as fake.To address this problem,Let the Discriminator know about this edge-case.More formally,Let the Discriminator look at the entire batch instead of single examplesIf there is lack of diversity,it will mark the examples as fakeThus,Generator will be forced to produce diverse samples.28November 27,2019Artificial28Artificial Intelligence29November 27,2019Mini-Batch GANsExtract features that capture diversity in the mini-batchFor e.g.L2 norm of the difference between all pairs from the batchFeed those features to the discriminator along with the imageFeature values will differ b/w diverse and non-diverse batchesThus,Discriminator will rely on those features for classificationThis in turn,Will force the Generator to match those feature values with the real dataWill generate diverse batchesArtificial Intelligence29Novem29Artificial Intelligence30Basic(Heuristic)Solutions Mini-Batch GANs Supervision with labelsNovember 27,2019Artificial IntelligenceBasic3031November 27,2019Artificial IntelligenceSupervision with Labels31November 27,2019Artificial3132November 27,2019Artificial IntelligenceAlternate view of GANs32November 27,2019Artificial3233November 27,2019Artificial IntelligenceAlternate view of GANs(Contd.)33November 27,2019Artificial3334November 27,2019Artificial IntelligenceEnergy-Based GANs34November 27,2019Artificial3435November 27,2019Artificial IntelligenceExamples35November 27,2019Artificial3536November 27,2019Artificial IntelligenceExamples36November 27,2019Artificial3637November 27,2019Artificial IntelligenceExamples37November 27,2019Artificial3738November 27,2019Artificial IntelligenceExamples38November 27,2019Artificial3839November 27,2019Artificial IntelligenceHow to reward Disentanglement?39November 27,2019Artificial39Artificial Intelligence40November 27,2019Recap:Mutual InformationMutual Information captures the mutual dependence between two variablesMutual information between two variables ,is defined as:Artificial Intelligence40Novem40Artificial Intelligence41November 27,2019InfoGANWe want to maximize the mutual information between and =(,)Incorporate in the value function of the minimax game.Artificial Intelligence41Novem41Artificial Intelligence42Conditional GANsNovember 27,2019Artificial IntelligenceCondit42Artificial Intelligence43November 27,2019Conditional GANsSimple modification to the originalGAN framework that conditions themodel on additional information forbetter multi-modal learning.Lends to many practicalapplications of GANs when wehave explicit supervision available.Artificial Intelligence43Novem43Artificial Intelligence44Conditional GANsNovember 27,2019Artificial IntelligenceCondit44Artificial Intelligence45November 27,2019Coupled GANLearning a joint distribution of multi-domain images.Using GANs to learn the joint distribution with samples drawn from the marginaldistributions.Direct applications in domain adaptation and image translation.Artificial Intelligence45Novem45Artificial Intelligence46Coupled GANNovember 27,2019Artificial IntelligenceCouple46Artificial Intelligence47Coupled GANNovember 27,2019Artificial IntelligenceCouple4748November 27,2019Artificial IntelligenceApplications48November 27,2019Artificial4849November 27,2019Artificial IntelligenceApplications49November 27,2019Artificial49Artificial Intelligence50Deep Convolution GANsNovember 27,2019Artificial IntelligenceDeep C50Artificial Intelligence51Deep Convolution GANsNovember 27,2019Artificial IntelligenceDeep C51Artificial Intelligence52Deep Convolution GANsNovember 27,2019Artificial IntelligenceDeep C52Artificial Intelligence53DCGAN(bedroom)November 27,2019Artificial IntelligenceDCGAN(53Artificial Intelligence54Image-to-ImageTranslationNovember 27,2019Artificial IntelligenceImage-54Artificial Intelligence55Image-to-ImageTranslationNovember 27,2019Artificial IntelligenceImage-55Artificial Intelligence56Text-to-ImageSynthesisNovember 27,2019Artificial IntelligenceText-t56Artificial Intelligence57Text-to-Image SynthesisNovember 27,2019Artificial IntelligenceText-t5758November 27,2019Artificial IntelligenceFace Aging with Conditional GANs58November 27,2019Artificial5859November 27,2019Artificial IntelligenceFace Aging with Conditional GANs59November 27,2019Artificial59Artificial Intelligence60November 27,2019Image Inpainting with GANsHaofeng Li,Guanbin Li,Liang Lin,Hongchuan Yu,and Yizhou Yu,“Context-Aware Semantic Inpainting”IEEE Transactions on Cybernetics(T-Cybernetics),DOI:10.1109/TCYB.2018.2865036,2018.Artificial Intelligence60Novem60Artificial Intelligence61GANs FutureNovember 27,2019Artificial IntelligenceGANs 61Artificial Intelligence62Explosion of GANNovember 27,2019Artificial IntelligenceExplos62
展开阅读全文