WebGL is a web standard for low-level 3D graphics API. It allows you to run your code directly on GPU, giving you all it's power. You write all ...
Allstreams
Development
Admin
Design
Management
Marketing
PopSci
SearchProfilePulltorefresh
ddone
8May2020at14:04WebGLtutorial:imageprocessingJavaScript*WebGL*
Sandbox
InthistutorialyouwilllearnhowtouseWebGLforimageprocessing.Wewillcoverbasicstufflikeinitialization,textureloading,andsimplefragmentshaders.IwilltrytocoverallaspectsofinteractionwithWebGLbutyoushouldhavedecentunderstandingofvanilajavascript.Ifyouwantmoreindepthexplanations,thereisagoodbookcalled"LearnWebGL",checkitout.
Hereareexamplesofwhatwewilldo:
Imagetwist
Imageblur
SOURCECODE
Ifyouwanttoskiptheoryandbuildsetupclickhere.
Theory
Everybodywhotriestoimplementgoodgraphicsquicklyunderstandshenowhassevereperformanceissues.Theamountofcomputationrequiredtoproduceanydecentsceneissimplyhuge.Youneedtoprocesseverypolygon(somemodelshavethousandsofthem)toprojectthemonscreen.Thenyouneedtorasterizethem.Applytextures,shading,reflectionsforeverypixelonthescreen.Andyoualsoneedtodoallofthisstuffatleast30timesinasecond.CPUsjustcan'thandlethisverywell.Especiallywhenusingsomescriptinglanguagewithlotsofoverheadforeveryoperation.
LuckilypeoplefoundasolutionandinventedGPUs.Allofthetasksdescribedabovearehighlyparallelintheirnature.Polygonsandpixelsusuallydonotdependoneachotherandcanbeeasily(andmuchmoreefficiently)processedatthesametime.GPUsareespeciallygoodatthis.Whilemodernprocessorsusuallyhave4-8cores,anydecentgraphicscardhasthousandsofthem.TheyaremuchlesscomplexthenCPUcoresandhighlyoptimizedforspecific3D-relatedcalculations.
WebGLisawebstandardforlow-level3DgraphicsAPI.ItallowsyoutorunyourcodedirectlyonGPU,givingyouallit'spower.YouwritealloftherenderingcodeinOpenGLShadingLanguageakaGLSL.It'snothardandverysimilartoC.ProgramswritteninGLSLusuallycalledshaders.TheyarecompiledandloadedintoagraphicscardinruntimeusingWebGLAPI.
Preparation
Technicallyyoudon'tneedtoinstallanything.Butwithoutaproperwebserveryouwon'tbeabletoloadimagesandadditionalscripts.Soit'sagoodideatohaveone.Iwillusewebpack-dev-server,it'seasytosetupanduse.
Firstthingthatyouneedtodoiscreateanemptyfolderandrunnpminitinside.YoucanskipallofthequestionsfromNPM.
Thenaddthislinestopackage.json
{
"scripts":{
"build":"webpack",
"serve":"webpack-dev-server"
},
"devDependencies":{
"copy-webpack-plugin":"^5.1.1",
"html-webpack-plugin":"^4.2.0",
"raw-loader":"^4.0.1",
"webpack":"^4.43.0",
"webpack-cli":"^3.3.11",
"webpack-dev-server":"^3.10.3"
}
}
Runnpminstallandcreateanewfilenamedwebpack.config.js.
Hereisthecontentsofwebpackconfig:
constpath=require('path');
constHtmlWebpackPlugin=require('html-webpack-plugin');
constwebpack=require('webpack');
constCopyWebpackPlugin=require('copy-webpack-plugin');
module.exports={
entry:'./index.js',
output:{
path:path.resolve(__dirname,'dist'),
filename:'index.js',
},
plugins:[
newHtmlWebpackPlugin({
template:'index.html'
}),
newCopyWebpackPlugin([{
from:"styles/*.css",
to:""
}])
],
mode:'development'
};
Nowyoucanstartdevserverbyrunningnpmrunserveandopenhttp://localhots:8080.Iwillnotexplainthisalldeeply,asthisisnotthemaintopic.Everythingshouldworkoutofthebox.
Code
Let'sdealwithHTMLrightaway.
Allweneedisacanvas,sohereitis.
index.html
Webgl
JustbasicHTMLtemplatewithacanvasandsliderthatwecanuse.
Now,it'stimetoinitializeWebGL.
index.js
//Usingwebpack'srawloadertogetshadercodeasJSstring.
//Muchmoreconvenientthanwritingthemdirectlyasstring
//orloadinginruntime
importvertfrom'!raw-loader!./vertex.glsl';
importfragfrom'!raw-loader!./fragment.glsl';
functionprepareWebGL(gl){
//Creatingandcompilingvertexshadr
letvertSh=gl.createShader(gl.VERTEX_SHADER);
gl.shaderSource(vertSh,vert);
gl.compileShader(vertSh);
//Thislineisveryimport
//Bydefaultifshadercompilationfails,
//WebGLwillnotshowyouerror,
//sodebuggingisalmostimpossible
console.log(gl.getShaderInfoLog(vertSh));
//Creatingandcompilingfragmentshader
letfragSh=gl.createShader(gl.FRAGMENT_SHADER);
gl.shaderSource(fragSh,frag);
gl.compileShader(fragSh);
//Thislineisveryimport
//Bydefaultifshadercompilationfails,
//WebGLwillnotshowyouerror,
//sodebuggingisalmostimpossible
console.log(gl.getShaderInfoLog(fragSh));
//LinkingprogramandpassingittoGPU
letprog=gl.createProgram();
gl.attachShader(prog,vertSh);
gl.attachShader(prog,fragSh);
gl.linkProgram(prog);
gl.useProgram(prog);
gl.viewport(0,0,c.width,c.height);
returnprog;
}
Errorchecksareomittedforclarity.So,howexactlyWebGLworks?It'ssortofaseparateworld.Youpasssomeprogramsanddatathere.ThenyouGPUexecutesyourprogramwithyourdataandgivesbackanimage.
Everyprogramconsistsoftwoparts:vertexandfragmentshaders.Vertexshaderisappliedtoeveryvertexorjustpointinspace,thatyoupassin.Hereyouperformall3Dstuffsuchastransformations,projections,andclipping.ThenGPUrasterizesyourshapes,whichmeansfillingthemwithpixelsakafragments.Actuallyfragmentsarenotexactlypixels.Butforthescopeofthistutorialtheycanbeusedinterchangeably.Afterrasterizationeveryfragmentispassedthroughfragmentshadertodetermineit'scolor.Finallyeverythingisdrawntotheframebufferanddisplayedonthescreen.
It'simportanttounderstandthatyourshadersareexecutedinparallel.Oneveryvertexandeveryfragmentindependently.Also,shadersproducevaluesnotbyreturningthembutsettingspecialvariables.Suchasgl_Positionandgl_FragColor,treatthemasreturnstatements.
Forthesakeofsimplicitywewillmostlyplaywithfragmentshadersandstayina2Dworld.
Hereissimplepass-throughvertexshader:
vertex.glsl
//Thisisourinputfromjsworld
attributevec2coords;
//Thisisoutputforthefragmentshader
//varyingvariablesarealittlespecial
//youwillseewhylater
varyinghighpvec2vTextureCoord;
voidmain(void){
//Textureandverticieshavedifferentcoordinatespaces
//wedothistoinvertYaxis
vTextureCoord=-coords;
//Settingvertixpositionforshapeassembler
//GLSLhasmanyconvenientvectorfunctions
//hereweextending2Dcoordsvectorto4Dwith2values
//0.0isaZcoordinate
//1.1isaW,specialvalueneededfor3Dmath
//justleaveit1fornow
gl_Position=vec4(coords,0.0,1.0);
}
Laterwewillfillourcanvaswitharectangle.Thisisneededtohavesomeplaneforapplyingtextures.Tounderstandhowfragmentshaderworksyouneedtocomprehendvaryingvariables.Let'simagineyouhavetwoverticesinaline.Invertexshaderyousetsomevaryingvariabletoredcolorfromoneofthemandtogreenfromanother.Allfragmentsbetweenthesetwopointswillgetdifferentvalueswhenreadingthisvaryingvariable.Itwillsmoothlytransitionfromredtogreen.Soifyousetfragmentcolortothevalueofthisvariableyouwillgetsomethinglikethis.Suchbehavioriscalledinterpolation.
Wewillalsouseacoupleofuniformvariables.Butnoneedtoworry,theyarequitesimple.Uniformvariablesarejustconstantparameters.Usefulforpassingglobalsettingandtextureids.
fragment.glsl
//Settingprecisionforfloatcalculations
precisionmediumpfloat;
//Thisisinputfromvertexshader
varyinghighpvec2vTextureCoord;
//Samplersareneedeedtoselecttextures
//actuallyitsintegers
uniformsampler2DuSampler;
//Thiswilltellushowmuchtoscrewtheimage
uniformfloatiter;
vec2coords;
floatx;
floaty;
floatl;
voidmain(void){
//Gettingdistancefromorigin
l=length(vTextureCoord);
//Justrenamingtoreducetyping
x=vTextureCoord[0];
y=vTextureCoord[1];
//Rotatingpointaroundorigin
coords[0]=x*cos(iter*l)-y*sin(iter*l);
coords[1]=x*sin(iter*l)+y*cos(iter*l);
//TransformingcoordinatesfromGLspacetotexturespace
//Allmathcanbedonedirectlytovectors
coords=coords/2.0-0.5;
//Fragmentshadermustsetthisvariable
gl_FragColor=texture2D(uSampler,coords);
}
Thekeyhereistounderstandthatangleofpointrotationisdependentonit'sdistancetothecenter.Thiswillresultinacooleffectoftexturetwistingandsuckinginablackhole.Ifwedeletethe*lpart,thewholethingwilljustrotateevenly.
Ourprogramisnowready,compiled,andloaded.Timetodealwiththedata.HereweareloadingvertexcoordinatestoWebGLmemory.Justtwotrianglestocoverthecanvassowehavesomesurfacefortexturing.BeawarethatWebGLcoordinatesworkdifferentfromcanvascoordinates.Unlikecanvas,it'soriginisatthecenterandallcoordinatesarenormalized.Sonomatterwhataspectratioyourcanvashas,XandYarealwaysin-1to1range.AlsoYcoordinateispointingupwards.
index.js
functionsetArrays(gl,prog){
//GettingWebGLbufferobject
letvertex_buffer=gl.createBuffer();
//Thisis2trianglesthatformasquare
//Eachtriangleconsistsof3points
//Eachpointconsistsoftwonumbers:XandYcoordinates
//GLcoordinatespacehasoriginincenter
//andspansfrom-1to1onbothaxes
//hereiswhyweneedtotransformourcoords
//infragmentshader
constvertices=[-1.0,-1.0,1.0,-1.0,-1.0,1.0,
1.0,1.0,1.0,-1.0,-1.0,1.0,
]
//LoadingourdataasARRAY_BUFFER
gl.bindBuffer(gl.ARRAY_BUFFER,vertex_buffer);
gl.bufferData(gl.ARRAY_BUFFER,newFloat32Array(vertices),gl.STATIC_DRAW);
//Findinglocationofvariable"coords"inGLmemory
//andbindingARRAY_BUFFERtoit
letcoord=gl.getAttribLocation(prog,"coords");
//VariablebindstolastbufferthatwaswrittentoGLmemory
gl.vertexAttribPointer(coord,2,gl.FLOAT,false,0,0);
gl.enableVertexAttribArray(coord);
//ARRAY_BUFFERisnowfreeandcanbereused
return[coord,vertex_buffer];
}
Arraydataisloaded.Timefortextures.
Bettertofindonethathassizesofpowersoftwo.
WebGLabletoautomaticallygeneratemipmapsforsuchtexturesandscalethemproperly.Atfirstwecreatesimple1x1bluepixeltextureandimmediatelyreturnit.Later,whenimageloads,wereplacethispixelwithpropertexturedata.
index.js
functionloadTexture(gl,prog,url){
//Creating1x1bluetuxture
consttexture=gl.createTexture();
constlevel=0;
constinternalFormat=gl.RGBA;
constwidth=1;
constheight=1;
constborder=0;
constsrcFormat=gl.RGBA;
constsrcType=gl.UNSIGNED_BYTE;
constpixel=newUint8Array([0,0,255,255]);//opaqueblue(RGBA)
//bindTextureworkssimilartobindBuffer
gl.bindTexture(gl.TEXTURE_2D,texture);
gl.texImage2D(gl.TEXTURE_2D,level,internalFormat,
width,height,border,srcFormat,srcType,
pixel);
//Loadingimage
constimage=newImage();
image.onload=function(){
gl.bindTexture(gl.TEXTURE_2D,texture);
gl.texImage2D(gl.TEXTURE_2D,level,internalFormat,
srcFormat,srcType,image);
//WebGL1hasdifferentrequirementsforpowerof2images
//vsnonpowerof2imagessocheckiftheimageisa
//powerof2inbothdimensions.
if(isPowerOf2(image.width)&&isPowerOf2(image.height)){
//Yes,it'sapowerof2.Generatemips.
gl.generateMipmap(gl.TEXTURE_2D);
}else{
//No,it'snotapowerof2.Turnoffmipsandset
//wrappingtoclamptoedge
gl.texParameteri(gl.TEXTURE_2D,gl.TEXTURE_WRAP_S,gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D,gl.TEXTURE_WRAP_T,gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D,gl.TEXTURE_MIN_FILTER,gl.LINEAR);
}
//Quikre-rendertodisplaynewtexture
//Seeimplementationbelow
render(0);
};
//Triggeringload
image.src=url;
returntexture;
}
functionisPowerOf2(value){
return(value&(value-1))==0;
}
Preparationsdone.Nowistimetotieeverythingtogether.
index.js
functionmain(){
//GettingWebGLcontextfromcanvas
constc=document.getElementById("c");
c.width=600;
c.height=600;
//Gettingslider
constrange=document.getElementById("range");
constgl=c.getContext("webgl");
constprog=prepareWebGL(gl);
constcoord=setArrays(gl,prog);
consttexture=loadTexture(gl,prog,"img.jpg");
//Handletocontrolamountoftwist
constiter=gl.getUniformLocation(prog,"iter");
constuSampler=gl.getUniformLocation(prog,'uSampler');
//Asissaidsamplersarejustintegers
//Telltheshadertousetexture0
gl.uniform1i(uSampler,0)
//Thisismainworkhorse
render=(it)=>{
//Bindingtexturetoslot0
gl.activeTexture(gl.TEXTURE0);
gl.bindTexture(gl.TEXTURE_2D,texture);
//Fillingscreenwithblackcolor
gl.clearColor(0.0,0.0,0.0,1.0);
gl.clear(gl.COLOR_BUFFER_BIT);
//Settingitertoslidervalue
gl.uniform1f(iter,it);
//Triggeringwebglrender
gl.drawArrays(gl.TRIANGLES,0,6);
}
render(0);
range.addEventListener("input",(e)=>{
render(e.target.value);
})
}
main()
That'sit.Youcantrytofollowalongandwriteityourself.Ifyoutotallystuckonsomething,theworkingsourcefromdemosaboveishere.
Bonus:Blurshader
GLSLisquiterestrictive.Forexampleyoucan'twriteloopswithnon-constantbounds.Thismayseemstrangeatfirst,butactuallyhavedecentreasoningbehindit.Mostoftherestrictionsareneededtohelpcompilersapplysomeaggressiveoptimizationtechniques.
So,hereistheblurshader.
fragment.glsl
varyinghighpvec2vTextureCoord;
uniformsampler2DuSampler;
precisionmediumpfloat;
uniformfloatiter;
uniformfloatuTextureSize;
voidmain(void){
floatpixel=1.0/uTextureSize;
vec2coords;
vec4color=vec4(0.0,0.0,0.0,0.0);
floatdiv=(iter+1.0)*(iter+1.0)*4.0;
for(inti=0;i<=100;i++){
if(float(i)>iter){
break;
}
for(intj=0;j<=100;j++){
if(float(j)>iter){
break;
}
coords=vTextureCoord.st/2.0-0.5;
coords+=vec2(float(i),float(j))*pixel;
color+=texture2D(uSampler,coords).rgba/div;
coords=vTextureCoord.st/2.0-0.5;
coords-=vec2(float(i),float(j))*pixel;
color+=texture2D(uSampler,coords).rgba/div;
}
inti2=-i;
for(intj=0;j<=100;j++){
if(float(j)>iter){
break;
}
coords=vTextureCoord.st/2.0-0.5;
coords+=vec2(float(i2),float(j))*pixel;
color+=texture2D(uSampler,coords).rgba/div;
coords=vTextureCoord.st/2.0-0.5;
coords-=vec2(float(i2),float(j))*pixel;
color+=texture2D(uSampler,coords).rgba/div;
}
}
gl_FragColor=vec4(color.rgb,1.0);
}
It'snotthebest,butitworks.AndIhopeyoucanlearnsomethingfromit.
Ideas
Youcanusethisprojectasatemplateforfutureexperiments.
Herearesomecoolideas:
SobelEdgedetection
Hueshift
Variuosphotofilters
Goodluckinyourprogrammingjourney.Tags:webgljavascriptimageprocessingblurHubs:
JavaScript
WebGL
Totalvotes3:↑3and↓0+3Views5.3KAddtobookmarks
5
5
Karma
0
Rating
@ddone
EmbeddedsoftwareengineerComments
Comments2
Youraccount
Login
Signup
Sections
Posts
News
Hubs
Companies
Authors
Sandbox
Information
Howitworks
Forauthors
Forcompanies
Documents
Agreement
Confidential
Services
Ads
Subscriptionplans
Content
Seminars
Megaprojects
FacebookTwitterTelegram
Languagesettings
About
Support
Returntooldversion
©2006–2022,Habr
